export_model

Export fine-tuned Unsloth models to GGUF, Ollama, vLLM, or Hugging Face formats for deployment, supporting quantization to optimize file size and performance.

Instructions

Export a fine-tuned Unsloth model to various formats

Input Schema

TableJSON Schema

Name	Required	Description
`model_path`	Yes	Path to the fine-tuned model
`export_format`	Yes	Format to export to (gguf, ollama, vllm, huggingface)
`output_path`	Yes	Path to save the exported model
`quantization_bits`	No	Bits for quantization (for GGUF export)

Implementation Reference

src/index.ts:552-661 (handler)

Main execution logic for the export_model tool. Parses arguments, generates format-specific Python scripts (using transformers and ctranslate2 for GGUF, transformers for HuggingFace), executes via executeUnslothScript, handles response.

          case 'export_model': {
            const {
              model_path,
              export_format,
              output_path,
              quantization_bits = 4,
            } = args as {
              model_path: string;
              export_format: 'gguf' | 'ollama' | 'vllm' | 'huggingface';
              output_path: string;
              quantization_bits?: number;
            };

            let script = '';
            
            if (export_format === 'gguf') {
              script = `
import json
import os
try:
    from transformers import AutoModelForCausalLM, AutoTokenizer
    import torch
    
    # Create output directory if it doesn't exist
    os.makedirs(os.path.dirname("${output_path}"), exist_ok=True)
    
    # Load the model and tokenizer
    model = AutoModelForCausalLM.from_pretrained("${model_path}")
    tokenizer = AutoTokenizer.from_pretrained("${model_path}")
    
    # Save the model in GGUF format
    from transformers import LlamaForCausalLM
    import ctranslate2
    
    # Convert to GGUF format
    ct_model = ctranslate2.converters.TransformersConverter(
        "${model_path}",
        "${output_path}",
        quantization="int${quantization_bits}"
    ).convert()
    
    print(json.dumps({
        "success": True,
        "model_path": "${model_path}",
        "export_format": "gguf",
        "output_path": "${output_path}",
        "quantization_bits": ${quantization_bits}
    }))
except Exception as e:
    print(json.dumps({"error": str(e), "success": False}))
`;
            } else if (export_format === 'huggingface') {
              script = `
import json
import os
try:
    from transformers import AutoModelForCausalLM, AutoTokenizer
    
    # Create output directory if it doesn't exist
    os.makedirs("${output_path}", exist_ok=True)
    
    # Load the model and tokenizer
    model = AutoModelForCausalLM.from_pretrained("${model_path}")
    tokenizer = AutoTokenizer.from_pretrained("${model_path}")
    
    # Save the model in Hugging Face format
    model.save_pretrained("${output_path}")
    tokenizer.save_pretrained("${output_path}")
    
    print(json.dumps({
        "success": True,
        "model_path": "${model_path}",
        "export_format": "huggingface",
        "output_path": "${output_path}"
    }))
except Exception as e:
    print(json.dumps({"error": str(e), "success": False}))
`;
            } else {
              return {
                content: [
                  {
                    type: 'text',
                    text: `Export format '${export_format}' is not yet implemented. Currently, only 'gguf' and 'huggingface' formats are supported.`,
                  },
                ],
                isError: true,
              };
            }
            
            const result = await this.executeUnslothScript(script);
            
            try {
              const exportResult = JSON.parse(result);
              if (!exportResult.success) {
                throw new Error(exportResult.error);
              }
              
              return {
                content: [
                  {
                    type: 'text',
                    text: `Successfully exported model to ${export_format} format:\n\n${JSON.stringify(exportResult, null, 2)}`,
                  },
                ],
              };
            } catch (error: any) {
              throw new Error(`Error exporting model: ${error.message}`);
            }
          }

src/index.ts:200-226 (registration)

Tool registration in MCP server.setTools(), defining name, description, and input schema for validation.

{
  name: 'export_model',
  description: 'Export a fine-tuned Unsloth model to various formats',
  inputSchema: {
    type: 'object',
    properties: {
      model_path: {
        type: 'string',
        description: 'Path to the fine-tuned model',
      },
      export_format: {
        type: 'string',
        description: 'Format to export to (gguf, ollama, vllm, huggingface)',
        enum: ['gguf', 'ollama', 'vllm', 'huggingface'],
      },
      output_path: {
        type: 'string',
        description: 'Path to save the exported model',
      },
      quantization_bits: {
        type: 'number',
        description: 'Bits for quantization (for GGUF export)',
      },
    },
    required: ['model_path', 'export_format', 'output_path'],
  },
},

src/index.ts:203-225 (schema)

Input schema defining parameters, types, descriptions, enum for export_format, and required fields.

inputSchema: {
  type: 'object',
  properties: {
    model_path: {
      type: 'string',
      description: 'Path to the fine-tuned model',
    },
    export_format: {
      type: 'string',
      description: 'Format to export to (gguf, ollama, vllm, huggingface)',
      enum: ['gguf', 'ollama', 'vllm', 'huggingface'],
    },
    output_path: {
      type: 'string',
      description: 'Path to save the exported model',
    },
    quantization_bits: {
      type: 'number',
      description: 'Bits for quantization (for GGUF export)',
    },
  },
  required: ['model_path', 'export_format', 'output_path'],
},

Unsloth MCP Server

export_model

Instructions

Input Schema

Implementation Reference

Other Tools

Latest Blog Posts

MCP directory API