xhs-comment-analyzer
Provides automated comment scraping and analysis for Xiaohongshu (Little Red Book) content, enabling brand marketing teams to search notes by brand and category, extract comments with sub-comments, perform keyword/sentiment/heat analysis, and export reports in Excel and JSON formats.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@xhs-comment-analyzer 搜索小熊电器小家电的评论并生成分析报告"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Xiaohongshu Comment Analysis Tool (XHS Comment Analyzer)
An automated UGC comment scraping and analysis tool for Xiaohongshu, designed for brand marketing teams. It supports searching notes by brand + category keywords, batch extracting comments (including sub-comments), three-dimensional analysis (keywords, sentiment, popularity), and exporting reports in Excel + JSON dual format. It runs as an MCP Server and can be directly invoked by AI clients such as TRAE / Claude / Cursor, and also supports standalone CLI operation.
Core Capabilities
Automatic Note Search: Search by brand + category keyword combination, auto-scroll to load more, intelligently filter irrelevant content.
Batch Comment Scraping: Open each note detail page one by one, extract complete information for parent and child comments, support resumable scraping.
Three-dimensional Analysis Engine: jieba + TF-IDF keyword extraction, sentiment dictionary + rule-based sentiment classification, interaction × time-decay popularity scoring.
Dual-format Report Export: 5-sheet Excel report + structured JSON data.
AI Workflow Integration: MCP Server exposes 4 tools, supporting natural language-driven full process.
Related MCP server: Xiaohongshu (RedBook) MCP Server
Quick Start
Installation
pip install -e .
playwright install chromiumRun Tests
python -m pytest tests/ -vCLI Usage
# 首次使用:检查登录状态(会打开浏览器,手动完成登录)
python run.py login
# 搜索并抓取评论(使用预设配置)
python run.py search
# 抓取单篇笔记评论
python run.py single "https://www.xiaohongshu.com/search_result/xxx?xsec_token=yyy"
# 对已有 JSON 重新分析
python run.py analyze data/output/report.jsonMCP Configuration
Add the following to the MCP configuration of TRAE / Claude / Cursor:
{
"mcpServers": {
"xhs-comment-analyzer": {
"command": "python",
"args": ["-m", "src.mcp_server"],
"cwd": "/path/to/xhs-comment-analyzer-package"
}
}
}After configuration, AI clients can be invoked via natural language: "Help me search for Xiaoxiong Electric small appliance comments and generate an analysis report."
MCP Tool List
Tool | Function |
| Batch search notes by brand + category, scrape comments, analyze, and export |
| Scrape and analyze comments from a single Xiaohongshu note |
| Re-run keyword/sentiment/popularity analysis on a scraped JSON file |
| Check Xiaohongshu login status |
Project Structure
xhs-comment-analyzer-package/
├── src/ # 源代码
│ ├── scraper/ # 抓取层
│ │ ├── browser.py # Playwright 浏览器管理 (登录态持久化、反检测)
│ │ ├── comment_scraper.py # 评论抓取核心 (搜索/单篇/批量/断点续抓)
│ │ └── models.py # 数据模型 (7 个 Pydantic 模型)
│ ├── analyzer/ # 分析层
│ │ ├── keywords.py # 关键词提取 (jieba + TF-IDF)
│ │ ├── sentiment.py # 情感分类 (词典 + 规则)
│ │ └── heat.py # 热度评分 (互动量 × 时效衰减)
│ ├── exporter/
│ │ └── excel_exporter.py # 导出 Excel (5 Sheet) + JSON
│ └── mcp_server.py # MCP Server (4 个工具)
├── tests/ # 单元测试 (38 个用例)
├── data/
│ ├── cookies/ # 登录 cookie 持久化
│ ├── dictionaries/ # 自定义词典
│ │ ├── domain_words.txt # 领域词典 (69 个小家电术语)
│ │ ├── stopwords.txt # 停用词表
│ │ ├── positive_words.txt # 正面情感词
│ │ ├── negative_words.txt # 负面情感词
│ │ ├── negation_words.txt # 否定词
│ │ └── degree_adverbs.txt # 程度副词 (词<TAB>权重)
│ └── output/ # 导出文件 (Excel/JSON)
├── docs/ # 产品文档
│ └── xhs-product-doc.html # 完整产品文档 (PRD/架构/工作流/算法/接口)
├── run.py # CLI 入口 (login/search/single/analyze)
├── conftest.py # pytest 配置
├── pyproject.toml # 依赖管理
├── .gitignore
└── README.mdOutput Format
Excel Report (5 Sheets)
Sheet | Content |
Comment Details | All comments sorted by popularity, including note title/URL/comment content/author/time/likes/replies/sentiment/popularity |
Analysis Summary | Number of scraped notes, total comments, product-related comments, sentiment distribution statistics |
Keywords Top10 | High-frequency keywords and their proportions |
Hot Comments Top10 | Top 10 comments by popularity score |
Note Summary | Likes, comments, and total popularity score of each note |
JSON Report
Structured full data, including task configuration, summary statistics, keyword list, complete comment list, and file path, convenient for programmatic consumption.
Core Algorithms
Keyword Extraction (jieba + TF-IDF)
Each comment is treated as an independent document. After jieba segmentation, stop words and single-character words are filtered out. TF-IDF weights are calculated (sklearn-style smoothing), returning the top 10 keywords and their proportions. A built-in small appliance domain dictionary (69 terms) ensures compound words are not split.
Sentiment Analysis (Dictionary + Rules)
Based on positive/negative sentiment dictionaries + negation word flipping (within a 2-word window, support double negation) + intensifier weighting ("very" ×1.5, "especially" ×2.0, etc.). Normalized to the range [-1, 1], mapped to positive/negative/neutral labels.
Popularity Score (Interaction × Time Decay)
base_score = like_count × 2 + reply_count × 3 + sub_comment_count × 1
time_decay = 0.95 ^ days_ago
heat_score = (base_score × time_decay / max_raw_heat) × 100Reply count has the highest weight (3) because replies indicate deep discussion; likes come second (2); sub-comments have the lowest weight (1). Decays 5% per day to ensure recent high-interaction comments rank higher.
Security Design
No Credential Bypass: The tool does not fill in account passwords, simulate QR code scanning, or automatically handle CAPTCHAs.
Human Intervention First: All login and CAPTCHA operations are performed manually by the user in a visible browser window.
Visible Browser: Always uses headless=False, allowing users to view and intervene at any time.
Risk Control Warning: Automatically stops after encountering CAPTCHAs 3 consecutive times to avoid triggering risk controls.
Resumable Scraping: Supports resume after interruption, progress saved in
progress.json.Cookie Persistence: Login state saved to
xhs_cookies.jsonto avoid frequent logins.
Tech Stack
Dependency | Purpose |
Python 3.12+ | Runtime |
Playwright | Browser automation |
MCP SDK | MCP Server protocol |
jieba | Chinese word segmentation |
openpyxl | Excel export |
Pydantic | Data model validation |
Product Documentation
Complete interactive product documentation is located at docs/xhs-product-doc.html. Open it in a browser to view. The documentation contains 9 chapters:
Product Overview
Product Requirements Document (PRD)
System Architecture
Workflow
Core Algorithms
Security and Anti-Detection
Data Model
MCP Interface
Usage Guide
Sample Data
The data/output/ directory contains sample output from an actual run (Xiaoxiong Electric small appliance category) for reference:
Notes: 8 (after filtering)
Comments: 58 (including sub-comments)
Sentiment Distribution: Positive 22.4%, Negative 10.3%, Neutral 67.2%
Keywords: Xiaoxiong, like, steamer
Configuration and Environment Variables
The tool runs without any environment variables or keys by default; all login states are performed manually via a visible browser, with cookies persisted to
data/cookies/.If integrating external services (proxies, API keys, etc.) in the future, please write the configuration to a
.envfile (which is ignored by.gitignore), and refer to the.env.exampletemplate. Never commit real keys.The following directories/files are excluded by
.gitignoreand will not be added to version control:data/cookies/*.json(login state),data/output/*(scraped results),data/progress.json,data/error.log,.env, etc.
Directory and File Description
Path | Committed? | Description |
| ✅ | All source code |
| ✅ | Unit tests |
| ✅ | Sentiment/segmentation dictionaries (text) |
| ❌ (only | Login cookies, sensitive |
| ❌ (only | Scraped and analysis results |
| ✅ | Product documentation |
| ❌ | Sensitive configuration |
License
This project is open source under the MIT License. See the LICENSE file for details (if not provided, contact the author to obtain it).
Available Tools
4 toolsanalyze_commentsA
对已抓取的评论JSON文件重新进行关键词/情感/热度分析。
适用于需要用不同参数重新分析已有数据的场景。
Args: file_path: 已抓取的评论JSON文件路径
Returns: JSON 格式的分析结果
Inputs: file_path (required). Returns: result
check_login_statusA
检查小红书登录状态,返回是否需要人工登录。
如果未登录,会自动打开浏览器窗口供用户手动登录。
Returns: JSON 格式的登录状态信息
Inputs: no named parameters. Returns: result
run_search_taskA
搜索小红书笔记并批量抓取产品评论,返回分析结果和文件路径。
自动化流程: 搜索匹配笔记 → 逐篇抓取评论 → 关键词提取 + 情感分类 + 热度评分 → 导出 Excel + JSON
Args: brand_keywords: 品牌词列表 (必须匹配), 如 ["小熊"] category_keywords: 品类词列表 (必须匹配), 如 ["小家电"] product_keywords: 具体产品词列表 (可选), 如 ["酸奶机", "蒸蛋器"] max_notes: 最多抓取笔记数 (1-200), 默认20 max_comments_per_note: 每篇最多评论数 (1-1000), 默认100 sort_by: 排序方式, general=综合, popularity=热门, time=最新 min_likes: 笔记最低点赞门槛, 默认0 comment_filter: 是否过滤产品相关评论, 默认True
Returns: JSON 格式的分析结果摘要,包含统计信息、关键词、情感分布和导出文件路径
Inputs: brand_keywords (required), category_keywords (required), sort_by, max_notes, +4 more. Returns: result
scrape_single_noteA
抓取单篇小红书笔记的评论并分析。
适用于已知笔记URL、只需抓取单篇评论的场景。
Args: note_url: 小红书笔记URL, 如 https://www.xiaohongshu.com/explore/xxxxx max_comments: 最多抓取评论数, 默认100
Returns: JSON 格式的评论列表和分析摘要
Inputs: note_url (required), max_comments. Returns: result
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 4 tools
Each tool targets a distinct phase of the workflow: run_search_task for full pipeline, scrape_single_note for single note scraping, analyze_comments for re-analysis, check_login_status for authentication. No overlap or ambiguity.
All tool names follow a consistent verb_noun pattern: run_search_task, scrape_single_note, analyze_comments, check_login_status. The verbs clearly indicate the action and nouns specify the target, making the naming predictable and clear.
With 4 tools, the set is tightly scoped to the server's purpose of comment analysis. Each tool serves a necessary and distinct role without excess. The count is appropriate for the domain.
The set covers the full workflow: login check, scraping (batch and single), and analysis. A minor gap is the lack of a dedicated tool for exporting or managing results beyond what the pipeline returns, but agents can work around this via file paths.
Related MCP Connectors
RedNote (Xiaohongshu, 小红书) note search, full notes, comment threads and creator profiles.
搜索笔记、浏览首页推荐、查看笔记内容与评论,并发表你的评论。直接在工作流中与小红书内容互动,高效跟进话题。
Public data from 20+ platforms, including Douyin, Xiaohongshu, Weibo and TikTok, as structured JSON
Social data for AI agents: TikTok, RedNote, Douyin, WeChat, X, YouTube and more. Pay per call.
Related MCP Servers
- AlicenseBqualityDmaintenanceEnables users to search and retrieve content from Xiaohongshu (Red Book) platform with smart search capabilities and rich data extraction including note content, author information, and images.130MIT
- FlicenseNot gradedqualityDmaintenanceEnables automated interaction with Xiaohongshu (Little Red Book) platform including searching posts, retrieving content and comments, and posting AI-generated comments with persistent login support.458-
- FlicenseNot gradedqualityDmaintenanceEnables automated searching and commenting on Xiaohongshu with AI-generated comments via MCP clients like Claude, supporting login persistence, note analysis, and four comment types.-
- AlicenseNot gradedqualityDmaintenanceEnables automated interaction and data scraping for Xiaohongshu (RedNote), including posting, liking, commenting, following, and retrieving user and note data.5MIT