Followin-MCP

Project Overview

This project can be understood as an event-level recommendation prototype for the crypto news domain:

• Upstream fetches raw content via Followin API + adapters.py

• normalizer.py standardizes raw content into structured ContentItem objects

• service.py performs explicit multi-path recall followed by semantic supplement in get_personal_feed

• clustering.py merges multiple items into an EventCluster representing the same event

• ranking.py generates a personalized feed using a multi-signal heuristic ranker + MMR rerank

Summary in one sentence:

A Followin MCP prototype for the crypto news domain: standardizes raw content into structured items, then produces a user-facing event-level feed through multi-path recall, semantic supplement, event clustering, and personalized ranking.

Related MCP server: onchainos MCP Server

Directory Structure

• followin_mcp/ Python package entry point

• followin_mcp/core/ Core business logic: adapter, model, normalizer, ranking, service

• followin_mcp/mcp/ MCP server entry point

• followin_mcp/demo/ Test agent and Web demo

• scripts/start_dev.sh Local one-click startup script

• web/ Frontend static assets

Core Modules

• followin_mcp/core/adapters.py Followin API adapter layer

• followin_mcp/core/models.py Data models

• followin_mcp/core/taxonomy_rules.py Taxonomy / rule configuration

• followin_mcp/core/normalizer.py Raw content standardization, entity extraction, event type identification

• followin_mcp/core/clustering.py Event clustering

• followin_mcp/core/ranking.py User recommendation ranking and explanation

• followin_mcp/core/semantic_recall.py Embedding indexing, semantic recall, item similarity

• followin_mcp/core/service.py Service entry point for MCP / application layer

Request Processing Sequence

sequenceDiagram
    participant U as MCP client / agent
    participant M as mcp/server.py
    participant S as service.py
    participant A as adapters.py
    participant F as Followin API
    participant N as normalizer.py
    participant SR as semantic_recall.py
    participant C as clustering.py
    participant R as ranking.py

    U->>M: call tool
    M->>S: get_latest / search / get_personal_feed ...

    S->>A: fetch raw content
    A->>F: HTTP request
    F-->>A: raw payload
    A-->>S: raw items

    loop per raw item
        S->>N: normalize(raw)
        N-->>S: ContentItem\nentities / event_type / scores
    end

    S->>SR: enqueue normalized items
    SR-->>S: background indexing

    opt personal feed / semantic supplement
        S->>SR: recall(query, candidate pool)
        SR-->>S: semantic candidates / similarity
    end

    opt personal feed
        S->>C: cluster_same_event(items)
        Note over C: uses item embedding similarity
        C-->>S: EventCluster list
        S->>R: rank_for_user(user, clusters)
        Note over R: uses semantic_match_score
        R-->>S: ranked clusters
    end

    S-->>M: tool payload
    M-->>U: MCP response

Processing Pipeline

1. MCP Entry Point

• mcp/server.py

  • Exposes 7 MCP tools

  • Converts MCP input parameters into service.py calls

  • Responsible for serializing ContentItem / EventCluster into MCP return structures

2. Raw Content Acquisition

• service.py -> adapters.py

  • get_latest_headlines / get_project_feed / get_project_opinions / get_trending_topics Directly passes through upstream pagination capabilities

  • get_trending_feeds / search_content Returns current snapshot results directly

  • get_personal_feed Performs explicit multi-path recall and semantic supplement first, then proceeds to event clustering and personalized ranking

3. Content Standardization

• normalizer.py

  • normalize(raw) converts a single raw content item into a ContentItem

  • Entity extraction sources:

    • tag

    • chain keyword

    • project alias

    • token alias

    • topic alias

    • dynamic alias (lightweight entity discovery / bypass NER)

  • Outputs:

    • projects / tokens / chains / topics

    • entity_sources

    • entity_confidence (strength of matching method)

    • event_type

    • credibility_score (source credibility)

    • importance_score (calculated via predefined rules)

4. Recall

• "Recall" here refers to:

  • Fetching a batch of candidate ContentItem objects from a larger content pool

  • Used for subsequent clustering, ranking, and pagination

• service.py performs explicit multi-path recall in the personal feed:

  • latest

  • trending

  • project

  • search

• Multi-path recall results are deduplicated by item.id

  • If the same item is hit by multiple paths, the version with stronger semantic match, higher importance, and more recent update time is prioritized

5. Event Clustering

• clustering.py

  • Input: list of ContentItem

  • Output: list of EventCluster

  • Clustering target is "the same event", not the same project or topic

  • Key signals:

    • Event type compatibility

    • Time window

    • Entity overlap with confidence weights

    • Title similarity

    • Optional semantic similarity

6. Personalized Ranking

• ranking.py

  • Input: EventCluster

  • Output: cluster feed ranked by user

  • Key signals:

    • importance_score

    • freshness_score

    • follow_affinity_score

    • interest_match_score

    • semantic_match_score

    • source_quality_score

    • risk_boost_score

  • mute_penalty

  • Final layer of MMR-style diversification rerank

7. Embedding

• semantic_recall.py

  • Normalized items are enqueued to an asynchronous embedding worker

  • Item embeddings are stored in a local SQLite semantic_index.db

• Embedding currently participates in three main tasks:

  • service.py performs query-aware semantic recall / semantic supplement in the personal feed based on the current query and candidate pool

  • clustering.py uses item embedding similarity as one of the clustering signals

  • ranking.py incorporates semantic matching signals into the final ranking via semantic_match_score

Personal Feed

• get_personal_feed is currently the only sessionized feed tool

• Main flow:

  • Explicit multi-path recall

  • Semantic supplement

  • Event clustering

  • Personalized ranking

• service.py maintains FeedSessionState internally, primarily storing:

  • pending_clusters

  • delivered_event_ids

  • delivered_item_ids

  • source_cursors

• Returned results include:

  • ranked_clusters

  • Expanded supporting items

  • next_cursor

  • has_more

• Pagination semantics:

  • The first request creates a feed session and attempts to fill pending_clusters with ranked clusters

  • The current page takes the first max_items clusters from the head of pending_clusters

  • Returned clusters / items are recorded in the delivered set to avoid repetition

  • Subsequent "more" requests prioritize consuming remaining pending_clusters

  • When the buffer falls below the refill threshold, another round of recall, semantic supplement, clustering, and ranking is triggered to replenish the buffer

Tool Semantics and Context Boundaries

These MCP tools can be divided into two categories:

• Content Query Tools

  • get_latest_headlines

  • get_trending_feeds

  • get_project_feed

  • get_project_opinions

  • get_trending_topics

  • search_content

• Recommendation Tools

  • get_personal_feed

Content query tools are stateless by default:

• get_latest_headlines / get_project_feed / get_project_opinions / get_trending_topics

  • Pass through upstream pagination capabilities

  • Return one page of results corresponding to the current request

• get_trending_feeds / search_content

  • Return current snapshot results directly

  • Do not maintain additional service-layer pagination state

get_personal_feed is currently the only stateful tool:

• service.py maintains a short-lived FeedSessionState

• Only exposes the feed session cursor externally

• Internally stores:

  • pending_clusters

  • delivered_event_ids

  • delivered_item_ids

  • source_cursors

• Subsequent "more" requests prioritize consuming pending_clusters within the session

Responsibility boundaries are roughly:

• Tool / Service layer

  • Provides content acquisition, candidate recall, clustering, ranking, and feed session management

• Agent layer

  • Responsible for tool selection and context maintenance in multi-turn conversations

  • E.g., continuing the previous batch, expanding the previous item, follow-up questions based on previous results

Current pagination semantics:

• get_latest_headlines / get_project_feed / get_project_opinions / get_trending_topics

  • Return upstream native cursor metadata

• get_personal_feed

  • Returns next_cursor and has_more

  • The semantics of cursor is feed session continuation, not standard list pagination

Technologies / Algorithms Used

• Python + MCP (FastMCP)

• Followin API adapter

• Rule-based entity extraction

  • alias matching

  • source tag matching

  • keyword rules

• Event classification

  • rule-based multi-signal scoring

• Entity confidence

  • strong / medium / weak

• Semantic recall

  • OpenAI embedding

  • cosine similarity

  • SQLite vector persistence

• Clustering

  • greedy incremental clustering

  • confidence-weighted Jaccard overlap

  • title Jaccard similarity

  • item embedding similarity

• Ranking

  • heuristic linear ranker

  • MMR-style diversification rerank

What the Prototype Lacks Compared to Industry Production Implementations

Recall / Recommendation System Capabilities

• The recommendation layer lacks true long-term / session user representation layering

• No asynchronous user profile update pipeline driven by behavioral logs

Content Understanding

• Currently has lightweight entity discovery / weak NER (tag, alias, rule, LLM-assisted extraction), but lacks a main pipeline for general online NER / entity linking with span offsets

• Event taxonomy is still a cold-start rule system, lacking calibration driven by labeled data

• Current credibility_score relies primarily on source type / metadata provided upstream; limited by data boundaries, it lacks finer source-level reliability and multi-source verification

Clustering

• Currently uses embedding similarity as one of the clustering signals, but it remains a heuristic multi-signal clustering approach, lacking a trained pairwise classifier / merge model

• Evolving into an online clustering system based on a persistent cluster store would require adding cluster assignment, as well as cluster lifecycle capabilities like cross-day evolution, splitting, and merging

Ranking

• Current ranking uses embedding-driven semantic_match_score, but the core remains a manually feature-weighted heuristic linear ranker rather than learned-to-rank

• The recommendation layer has not integrated behavioral features like clicks, dwell time, or shares

• The recommendation layer currently only performs light MMR diversification, lacking more systematic quota control and business constraints

Currently Exposed MCP Tools

• get_latest_headlines

• get_trending_feeds

• get_project_feed

• get_project_opinions

• get_trending_topics

• search_content

• get_personal_feed

Web Demo

If you want to test "random user profile + multi-turn conversation" more intuitively, you can start a Web demo:

python3 -m followin_mcp.demo.webapp

Or after installation:

followin-mcp-web

Then open:

http://127.0.0.1:8000

This demo supports:

• Random user profile generation

• Creating a LangChain agent session for the current profile

• Retaining multi-turn chat context within the same session

• Retaining the latest available next_cursor context for tools that support pagination

• Displaying one or more actual tool calls that occurred in each turn

• Displaying tool parameters and return result cards

Please ensure the following is configured in .env before running:

FOLLOWIN_API_KEY=your_api_key
OPENAI_API_KEY=your_openai_api_key

MCP Server

The following capabilities are currently exposed as MCP tools:

• get_latest_headlines

• get_trending_feeds

• get_project_feed

• get_project_opinions

• get_trending_topics

• search_content

• get_personal_feed

Startup method:

python3 -m followin_mcp.mcp.server

Or use after installation:

followin-mcp-server

ACP Agent

If you want to expose the current chat agent as an ACP stdio agent to clients that support ACP, you can start:

python3 -m followin_mcp.acp.server

Or use after installation:

followin-acp-agent

The current ACP wrapper directly reuses FollowinChatAgent:

• ACP session corresponds to a FollowinChatAgent session

• prompt() internally calls the existing chat_stream()

• Assistant text is passed back to the ACP client in streaming chunks

• User profile is read from FOLLOWIN_ACP_PROFILE_JSON by default, or uses a built-in default profile if not configured

If you are connecting to a local acpx / OpenClaw, you can start with a minimal configuration in ~/.acpx/config.json:

{
  "agents": {
    "followin": {
      "command": "/path/to/python",
      "args": ["-m", "followin_mcp.acp.server"],
      "cwd": "/path/to/followin-mcp"
    }
  }
}

No extra env writing is needed here:

• followin_mcp.acp.server will load_dotenv() upon startup

• As long as cwd points to the project root, it will read the .env in the repository