title: Arm Pulse emoji: ⚡ colorFrom: blue colorTo: indigo sdk: docker pinned: false

⚡ Arm Pulse - Arm64 LLM Inference Benchmark Suite + MCP Server

KleidiAI-optimized LLM benchmarking and inference server for Arm64 cloud infrastructure. Built for the Arm AI Optimization Challenge 2026.

Related MCP server: Local AI MCP

🎯 What is Arm Pulse?

Arm Pulse is a one-command benchmarking tool that:

1. Deploys LLMs (Llama 3.2) on Arm64 cloud instances using llama.cpp + KleidiAI

2. Measures real performance tokens/sec, time-to-first-token, memory usage across quantization levels (Q4_K_M vs Q8_0)

3. Serves results via an MCP-compatible FastAPI server any agent framework can call

4. Visualizes everything in a clean real time dashboard

🏗️ Architecture

arm-pulse/

├── benchmark/ # llama.cpp + KleidiAI inference engine + metrics

├── mcp_server/ # FastAPI MCP-compatible LLM endpoint

├── dashboard/ # Real-time results dashboard (HTML)

├── scripts/ # One-command setup + benchmark + server scripts

└── docker/ # Arm64-optimized Docker configuration

🚀 Quick Start (Arm64 Instance)

1. Clone and setup

git clone https://github.com/sirmos/arm-pulse.git
cd arm-pulse
bash scripts/setup.sh

2. Run benchmark

bash scripts/run_benchmark.sh

3. Start MCP server

bash scripts/start_mcp.sh

4. Open dashboard

Navigate to http://your-instance-ip:8000 in your browser.

☁️ Tested Arm64 Platforms

📊 What We Benchmark

Models

🔌 MCP Server API

Example: Generate

curl -X POST http://localhost:8000/generate \
  -H "Content-Type: application/json" \
  -d '{"prompt": "What is KleidiAI?", "model": "Llama-3.2-3B-Q4_K_M"}'

⚙️ Arm-Specific Optimizations

• KleidiAI: Arm's optimized kernel library for ML workloads

• llama.cpp Arm SVE: Scalable Vector Extension support enabled at build time

• Native CPU tuning: -DLLAMA_NATIVE=ON compiles for exact CPU microarchitecture

• Thread optimization: Automatically uses all available Arm cores

📄 License

MIT License - see LICENSE

Built for the Arm AI Optimization Challenge 2026