A
licenseNot graded
qualityB
maintenanceEnables clients to optimize prompt prefix caching by matching longest common token prefixes with a radix trie, eliminating redundant prefill computation and reducing time-to-first-token.
7
MIT
genpark-dynamic-prompt-prefix-cache-skill
by Alpha-Park
Related Servers
Alternatives to genpark-dynamic-prompt-prefix-cache-skill
No user-submitted related servers found.
Related Servers
- AlicenseNot gradedqualityBmaintenanceEnables verification of draft-target speculative decoding with rejection sampling, acceptance-rate telemetry, and dynamic speedup estimation, alongside edge inference primitives such as INT8 quantization, paged KV allocation, and radix prefix caching.8MIT
- AlicenseNot gradedqualityBmaintenanceEnables MCP clients to manage LLM inference memory through virtual paged-attention KV-cache block mapping, non-contiguous physical page allocation, zero-copy fragmentation tracking, radix-trie prefix caching, INT8 quantized compute, and speculative decoding verification. It also exposes prefill and decode latency telemetry so edge deployments can be benchmarked and tuned without external dependencies.7MIT
- AlicenseNot gradedqualityBmaintenanceEnables zero-dependency INT8 symmetric quantization of LLM weights and activations on a per-tensor or per-channel basis, reporting SNR and MSE reconstruction telemetry along with TTFT/TPOT latency and jitter metrics. Also exposes edge inference primitives such as PagedAttention block allocation, radix prefix caching, and speculative decoding verification through the Model Context Protocol.7MIT
- AlicenseNot gradedqualityBmaintenanceEnables profiling and optimizing edge and on-device LLM inference by measuring TTFT, TPOT, tokens/second throughput, and P50/P90/P99 jitter while providing standard-library primitives for quantization, paged attention, speculative decoding, and prefix caching.7MIT
- AlicenseAqualityAmaintenanceReal-time cluster health monitoring, pre-request NLMS latency prediction, and intelligent prompt routing across multi-instance LLM backends (vLLM, Ollama, SGLang, TGI).43Apache 2.0
This server cannot be deployed
Maintenance
ActivityMaintained
ResponsivenessNo issues