llama.cpp v1.0.0 Hardened Server (OCP FP8 KV Cache, MoE Host Cache & LFM3) Launcher (run-llama-cpp-v1-fp8-moe.sh)
Launches llama-server (v1.0.0+ SemVer) with OCP FP8 (E4M3) KV cache, FlashAttention, configurable MoE CPU host-memory caching, and SSRF/DNS-rebinding-hardened OpenAI-compatible endpoints.
llama.cpp v1.0.0 Hardened Server (OCP FP8 KV Cache, MoE Host Cache & LFM3) Launcher
Launches llama-server (v1.0.0+ SemVer) with OCP FP8 (E4M3) KV cache, FlashAttention, configurable MoE CPU host-memory caching, and SSRF/DNS-rebinding-hardened OpenAI-compatible endpoints.
#!/usr/bin/env bash
# Tinker AI — llama.cpp v1.0.0 OCP FP8 KV Cache & MoE Host-Memory Cache Launcher (Guide #51)
set -euo pipefail
MODEL="models/LiquidAI_LFM3-1.7B-Q4_K_M.gguf"
KV_TYPE="fp8_e4m3"
CTX="32768"
CPU_MOE="0"
PORT="8080"
echo "=== Verifying llama.cpp v1.0.0+ binary & starting hardened llama-server ==="
llama-server --version || true
EXTRA_FLAGS=()
if [ "$CPU_MOE" -gt 0 ] 2>/dev/null; then
EXTRA_FLAGS+=(--n-cpu-moe "$CPU_MOE")
fi
# Note: llama.cpp v1.0.0 enables SSRF/DNS-rebinding hardening & auto-cancels MCP calls on disconnect
exec llama-server \
-m "$MODEL" \
-c "$CTX" \
-fa on \
--cache-type-k "$KV_TYPE" \
--cache-type-v "$KV_TYPE" \
-ngl 99 \
"${EXTRA_FLAGS[@]}" \
--host 127.0.0.1 \
--port "$PORT"
Customizable Parameter Reference for run-llama-cpp-v1-fp8-moe.sh
This SH artifact exposes 5 template variables compiled client-side in your browser.
| Template Token | Parameter Description | Verified Default | Example Format |
|---|---|---|---|
{{GGUF_MODEL_PATH}} | GGUF Model Path (e.g. LFM3-1.7B or Qwen3.5-35B-A3B) | models/LiquidAI_LFM3-1.7B-Q4_K_M.gguf | models/LiquidAI_LFM3-1.7B-Q4_K_M.gguf |
{{KV_CACHE_TYPE}} | KV Cache Precision (f16 | q8_0 | fp8_e4m3) | fp8_e4m3 | fp8_e4m3 |
{{CTX_SIZE}} | Context Window (-c) | 32768 | 32768 |
{{N_CPU_MOE}} | MoE Layers Kept in Host RAM Cache (0 for dense) | 0 | 0 |
{{SERVER_PORT}} | HTTP Server Port | 8080 | 8080 |
Verified Environment for "llama.cpp v1.0.0 (OCP FP8 KV Cache & MoE Host-Memory Caching) vs SGLang Consortium: Running Liquid AI LFM3 (256K Ctx) & Kimi-K3.1-Pro Across CUDA, Vulkan, SYCL & Hexagon NPUs"
Workstation / Desktop: NVIDIA RTX 5090 (32GB Blackwell), Apple Mac mini M5 Pro (64GB Unified), Intel Arc Pro B60 (24GB SYCL), Snapdragon X2 Elite (OpenCL Adreno / Hexagon NPU) | Cluster Reference: 16×141GB+ H200/B200 or NVLink Fusion GPU+NPU racks for Kimi-K3.1-Pro
v1.0.0 & ggml v1.0.0 (SemVer C-API, 35+ legacy llama.h functions pruned, RPC protocol v9)SGLang Consortium stack (Nov 12, 2026) & NVIDIA/Rebellions NVLink Fusion disaggregated inference referenceModels: LiquidAI/LFM3-1.7B (256K ctx, ~1.05 GB Q4), LiquidAI/LFM3-VL-2B (128K ctx), LiquidAI/LFM3-Omni-2B (2.2B), moonshotai/Kimi-K3.1-Pro (2.8T total / 104B active, MXFP4)llama-server)Once GGUF/safetensors weights are cached locally, llama-server v1.0.0 runs completely offline with built-in SSRF, CORS, and DNS-rebinding protections.
- RPC Protocol v9 Breaking Upgrade:
llama.cpp v1.0.0removed legacyDIAG_MASK_INF/DIAG_MASK_ZEROops and bumped theggml-rpcprotocol to v9. Every distributed worker node must be upgraded tov1.0.0+simultaneously—mixingv1.0.0clients with pre-1.0rpc-servernodes fails immediately. - 35+ Pruned C-API Symbols in
llama.h: Downstream C/C++ bindings (custom Rust/Python/Go wrappers) calling deprecated functions likellama_set_warmupor legacy defragmentation-threshold setters will fail to compile againstlibllama v1.0.0until updated to the clean1.xheader. Kimi-K3.1-ProIs Strictly Multi-Node Cluster Territory (~1.56 TBMXFP4): Even with nativeMXFP4weights andMXFP8activations,Kimi-K3.1-Pro(2.8Ttotal,104Bactive across 93 layers) spans 96 safetensors shards (~1.56 TB)—requiring16× 141GB+HBM GPUs or high-RAM multi-node NVLink/RDMA clusters rather than a single workstation.
3-Step Verification Checklist After Deploying run-llama-cpp-v1-fp8-moe.sh
- Verified
llama-server --versionisv1.0.0+and upgraded all distributedrpc-servernodes together to RPC protocol v9. - Enabled
-fa on --cache-type-k fp8_e4m3 --cache-type-v fp8_e4m3(orq8_0) for 32K+ context workloads. - Bound
llama-serverto127.0.0.1behind a reverse proxy (never exposing raw--host 0.0.0.0without auth) to complement v1.0.0 SSRF/DNS-rebinding protections.