info

Instructions

Implementation Reference

• mcp_turboquant/server.py:37-86 (handler)

  The `info` tool handler — registered via @mcp.tool() decorator. Calls get_model_info() and returns model metadata (architecture, parameters, size, context length).

  @mcp.tool()
  def info(model: str) -> dict[str, Any]:
      """Get model info from HuggingFace — parameters, size, architecture.
  
      Lightweight call using the HuggingFace API. No GPU or heavy
      dependencies required.
  
      Args:
          model: HuggingFace model ID (e.g. 'meta-llama/Llama-3.1-8B-Instruct')
                 or local path to a model directory.
  
      Returns:
          Model metadata including architecture, parameter count, size,
          hidden dimensions, number of layers, vocabulary size, and
          context length.
      """
      result = get_model_info(model)
  
      if not result.get("found"):
          return {
              "error": f"Model not found: {result.get('error', 'unknown')}",
              "model": model,
          }
  
      # Build a clean response (strip internal fields like raw config)
      output = {
          "model": result.get("model_id", result.get("source")),
          "found": True,
          "architecture": result.get("arch", "unknown"),
          "parameters": result.get("params_human", "unknown"),
          "size": result.get("size_human", "unknown"),
          "size_bytes": result.get("size_bytes", 0),
          "hidden_size": result.get("hidden_size", 0),
          "num_layers": result.get("num_layers", 0),
          "vocab_size": result.get("vocab_size", 0),
          "context_length": result.get("context_length", 0),
      }
  
      if result.get("local"):
          output["local"] = True
  
      # Add compression estimates
      if result.get("size_bytes"):
          sz = result["size_bytes"]
          output["estimated_sizes"] = {
              "4bit": format_size(sz / estimate_compression(16, 4)),
              "8bit": format_size(sz / estimate_compression(16, 8)),
          }
  
      return output

• mcp_turboquant/server.py:38-52 (schema)

  Docstring of the `info` tool serving as input schema — accepts a single 'model' string (HuggingFace model ID or local path).

  def info(model: str) -> dict[str, Any]:
      """Get model info from HuggingFace — parameters, size, architecture.
  
      Lightweight call using the HuggingFace API. No GPU or heavy
      dependencies required.
  
      Args:
          model: HuggingFace model ID (e.g. 'meta-llama/Llama-3.1-8B-Instruct')
                 or local path to a model directory.
  
      Returns:
          Model metadata including architecture, parameter count, size,
          hidden dimensions, number of layers, vocabulary size, and
          context length.
      """

• mcp_turboquant/server.py:37-37 (registration)

  Tool registered via @mcp.tool() decorator on the FastMCP instance named 'TurboQuant'.

  @mcp.tool()

• mcp_turboquant/model_info.py:113-215 (helper)

  The `get_model_info()` helper function — fetches model info from HuggingFace Hub or local path, extracts architecture, parameter count, size, context length.

  def get_model_info(model_id_or_path: str) -> dict[str, Any]:
      """Get model information from HuggingFace or local path."""
      info: dict[str, Any] = {"source": model_id_or_path}
  
      try:
          from huggingface_hub import hf_hub_download, model_info as hf_model_info
  
          mi = hf_model_info(model_id_or_path)
          info["model_id"] = mi.id
          info["size_bytes"] = sum(
              s.size
              for s in mi.siblings
              if s.rfilename.endswith((".safetensors", ".bin")) and s.size is not None
          )
          info["size_human"] = format_size(info["size_bytes"])
  
          # Try to get parameter count from config
          config_path = hf_hub_download(model_id_or_path, "config.json")
          with open(config_path) as f:
              config = json.load(f)
          info["config"] = config
          info["arch"] = config.get("architectures", ["unknown"])[0]
          info["hidden_size"] = (
              config.get("hidden_size")
              or config.get("n_embd")
              or config.get("d_model")
              or 0
          )
          info["num_layers"] = (
              config.get("num_hidden_layers")
              or config.get("n_layer")
              or config.get("num_layers")
              or 0
          )
          info["vocab_size"] = config.get("vocab_size", 0)
          info["context_length"] = (
              config.get("max_position_embeddings")
              or config.get("n_positions")
              or config.get("max_seq_len")
              or config.get("seq_length")
              or 0
          )
  
          # Estimate parameters
          h = info["hidden_size"]
          n = info["num_layers"]
          v = info["vocab_size"]
          if h and n and v:
              params = 12 * n * h * h + v * h
              info["params_estimate"] = params
              info["params_human"] = (
                  f"{params / 1e9:.1f}B" if params > 1e9 else f"{params / 1e6:.0f}M"
              )
  
          # If HF API didn't return file sizes, estimate from parameters
          if not info["size_bytes"] and info.get("params_estimate"):
              info["size_bytes"] = info["params_estimate"] * 2  # FP16
              info["size_human"] = format_size(info["size_bytes"]) + " (estimated)"
  
          info["found"] = True
      except Exception as e:
          # Check if local path
          if os.path.isdir(model_id_or_path):
              info["found"] = True
              info["local"] = True
              total = sum(
                  os.path.getsize(os.path.join(dp, f))
                  for dp, _, fns in os.walk(model_id_or_path)
                  for f in fns
                  if f.endswith((".safetensors", ".bin"))
              )
              info["size_bytes"] = total
              info["size_human"] = format_size(total)
              local_config = os.path.join(model_id_or_path, "config.json")
              if os.path.exists(local_config):
                  with open(local_config) as f:
                      config = json.load(f)
                  info["config"] = config
                  info["arch"] = config.get("architectures", ["unknown"])[0]
                  info["hidden_size"] = (
                      config.get("hidden_size")
                      or config.get("n_embd")
                      or config.get("d_model")
                      or 0
                  )
                  info["num_layers"] = (
                      config.get("num_hidden_layers")
                      or config.get("n_layer")
                      or config.get("num_layers")
                      or 0
                  )
                  info["vocab_size"] = config.get("vocab_size", 0)
                  info["context_length"] = (
                      config.get("max_position_embeddings")
                      or config.get("n_positions")
                      or config.get("max_seq_len")
                      or 0
                  )
          else:
              info["found"] = False
              info["error"] = str(e)
  
      return info

• mcp_turboquant/model_info.py:12-18 (helper)

  Helper `format_size()` — converts bytes to human-readable string, used by the info tool to format model size.

  def format_size(size_bytes: int | float) -> str:
      """Format bytes to human-readable string."""
      for unit in ["B", "KB", "MB", "GB", "TB"]:
          if size_bytes < 1024:
              return f"{size_bytes:.1f} {unit}"
          size_bytes /= 1024
      return f"{size_bytes:.1f} PB"