Skip to main content
Glama
Sign In
deenesjakoruzh

TokenScope

by cdgaete
GitHub
Python
MIT License
1
  • Linux
  • Apple
OverviewInspectNewSchemaRelated ServersReviewsScore
Need Help?View Source CodeReport Issue

Integrations

  • Supports .gitignore files when scanning directories to automatically filter out files that are excluded from version control.

TokenScope

Token-Aware Directory Explorer for Large Language Models (LLMs).

A Model Context Protocol (MCP) server that helps LLMs efficiently explore and understand codebases and directory structures.

Overview

TokenScope provides intelligent directory structure analysis and token-aware file content exploration designed for LLMs like Claude. It helps LLMs understand codebases by:

  1. Exploring directory structures with token-aware summarization
  2. Viewing file contents with token limitations in mind
  3. Generating comprehensive reports about directories

Key Features

Token-Aware Directory Exploration

  • Automatic summarization for large directories while showing small directories in full
  • Respect for token limits to maximize useful information within constraints
  • Built-in security with base path validation
  • Smart filtering with default patterns and .gitignore support
  • Accurate directory statistics for even the largest directories

Simple, Intuitive Tools

TokenScope provides just three core tools:

  1. explore_directory - Scan and understand directory structures
  2. view_content - Access file contents with token awareness
  3. generate_report - Create comprehensive reports (with option to save to file)

Installation

Prerequisites

  • Python 3.10 or higher
  • uv (recommended for dependency management)

Installation (PyPI)

This is the recommended method for most users who just want to use TokenScope:

# Install from PyPI using uv (recommended) uv pip install tokenscope

Running TokenScope

The --base-path argument is mandatory for security reasons. It restricts all file operations to the specified directory.

# Run using the installed package uv run --with tokenscope tokenscope --base-path /path/to/allowed/directory

Configuring in Claude Desktop

  1. Locate Claude Desktop's configuration file (typically ~/.config/claude/config.json)
  2. Add TokenScope to the mcpServers section:
"mcpServers": { "TokenScope": { "command": "uv", "args": [ "run", "--with", "tokenscope", "tokenscope", "--base-path", "/your/secure/base/path" ] } }
  1. Replace /your/secure/base/path with the directory you want to restrict operations to
  2. Save the configuration file and restart Claude Desktop

Usage

Running TokenScope Server

The --base-path argument is required for security (it restricts file operations to the specified directory):

tokenscope --base-path /path/to/allowed/directory

Testing Tools Directly

TokenScope includes a test mode for trying out tools directly:

# Test directory exploration tokenscope --base-path /path/to/allowed/directory --test "explore:/path/to/directory" # Test with custom ignore patterns tokenscope --base-path /path/to/allowed/directory --test "explore:/path/to/directory?ignore=cache,*.log,tmp/&gitignore=false" # Test file viewing tokenscope --base-path /path/to/allowed/directory --test "view:/path/to/file" # Test report generation tokenscope --base-path /path/to/allowed/directory --test "report:/path/to/directory" # Test report generation with file output and custom ignore patterns tokenscope --base-path /path/to/allowed/directory --test "report:/path/to/directory?ignore=*.bak,temp/ > /path/to/output.md"

Example Prompts

Here are some examples of how to use TokenScope with Claude:

Could you explore my project directory at /path/to/project and tell me about its structure?
Can you show me the content of the file at /path/to/file.py?
Please generate a comprehensive report about my project at /path/to/project and save it to /path/to/report.md

Accurate Directory Statistics

TokenScope now provides two levels of directory information:

  1. Quick Scan Statistics: Information about files and directories visible in the tree view
  2. Full Directory Statistics: Complete counts of ALL files and directories, even in very large repositories

This dual approach ensures that even for massive directories (with thousands or millions of files), you'll get accurate information about the total number of files, directories, and disk usage. This is especially valuable when working with large codebases where a complete directory listing would exceed token limits.

Example Output

QUICK SCAN STATISTICS (files visible in tree): Files shown in tree: 47 Directories shown in tree: 16 Size shown in tree: 185.9 MB FULL DIRECTORY STATISTICS (all files): Total files: 16,059 Total directories: 8 Total disk size: 2.1 GB

Smart Filtering with Ignore Patterns

TokenScope automatically filters out common directories and files that typically don't contribute to understanding a codebase:

  • Default ignored patterns: .git/, .venv/, venv/, __pycache__/, node_modules/, build/, dist/, etc.
  • Custom ignore patterns: You can specify additional patterns to ignore via the ignore_patterns parameter
  • .gitignore support: TokenScope can automatically respect .gitignore files in the directories it scans

This functionality helps prevent token waste on irrelevant files and directories like:

  • Dependency directories (e.g., node_modules, virtual environments)
  • Build artifacts and cache directories
  • Version control metadata
  • IDE configuration files

Using Ignore Patterns in CLI Test Mode

# Ignore specific patterns tokenscope --base-path /path --test "explore:/code?ignore=*.log,temp/" # Disable .gitignore processing tokenscope --base-path /path --test "explore:/code?gitignore=false" # Both together tokenscope --base-path /path --test "explore:/code?ignore=*.tmp&gitignore=false"

Security Features

TokenScope includes important security features:

  • All file operations are validated to ensure they're within the specified base directory
  • Attempts to access files outside the base path are rejected
  • The base path is set once when starting the server and cannot be changed without restart

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

This server cannot be installed

-
security - not tested
A
license - permissive license
-
quality - not tested

local-only server

The server can only run on the client's local machine because it depends on local resources.

A Model Context Protocol server that enables token-aware directory exploration and file analysis for LLMs, helping them understand codebases through intelligent scanning and reporting.

  1. Overview
    1. Key Features
      1. Token-Aware Directory Exploration
      2. Simple, Intuitive Tools
    2. Installation
      1. Prerequisites
      2. Installation (PyPI)
    3. Running TokenScope
      1. Configuring in Claude Desktop
    4. Usage
      1. Running TokenScope Server
      2. Testing Tools Directly
    5. Example Prompts
      1. Accurate Directory Statistics
        1. Example Output
      2. Smart Filtering with Ignore Patterns
        1. Using Ignore Patterns in CLI Test Mode
      3. Security Features
        1. License
          1. Acknowledgments

            Related MCP Servers

            • MCP File Context Server

              bsmi021
              -
              security
              A
              license
              -
              quality
              A Model Context Protocol server that enables LLMs to read, search, and analyze code files with advanced caching and real-time file watching capabilities.
              Last updated -
              45
              8
              JavaScript
              MIT License
              • Linux
              • Apple

            • Python MCP Server

              hesiod-au
              -
              security
              A
              license
              -
              quality
              A Model Context Protocol server that extracts and analyzes Python code structures, focusing on import/export relationships between files to help LLMs understand code context.
              Last updated -
              4
              Python
              MIT License

            • Unstructured Document Processor MCP

              MKhalusova
              -
              security
              F
              license
              -
              quality
              A Model Context Protocol server that enables LLMs to extract and use content from unstructured documents across a wide variety of file formats.
              Last updated -
              2
              Python
              • Apple

            • MCP Filesystem Server

              safurrier
              -
              security
              A
              license
              -
              quality
              A Model Context Protocol server that provides secure and intelligent interaction with files and filesystems, offering smart context management and token-efficient operations for working with large files and complex directory structures.
              Last updated -
              5
              Python
              MIT License
              • Apple
              • Linux

            View all related MCP servers

            Appeared in Searches

            New MCP Servers

            ID: h3cyyvuwfr