GMKtec EVO-X5 Pro (192GB / 160GB UMA) + Unsloth XDNA 2 NPU (Q4NX) & NVIDIA Nemotron 3.5 Super VL 120B Runner (run-evox5-unsloth-npu-nemotron35.sh)
Audits 160GB Radeon 8065S UMA & /dev/accel/accel0 XDNA 2 NPU on GMKtec EVO-X5 Pro (Ryzen AI Max+ PRO 495), runs NVIDIA Nemotron-3.5-Super-VL-120B-A12B or Lightning 30B with configurable reasoning budgets, and pairs Unsloth Desktop Lemonade/FastFlowLM Q4NX NPU inference.
GMKtec EVO-X5 Pro (192GB / 160GB UMA) + Unsloth XDNA 2 NPU (Q4NX) & NVIDIA Nemotron 3.5 Super VL 120B Runner
Audits 160GB Radeon 8065S UMA & /dev/accel/accel0 XDNA 2 NPU on GMKtec EVO-X5 Pro (Ryzen AI Max+ PRO 495), runs NVIDIA Nemotron-3.5-Super-VL-120B-A12B or Lightning 30B with configurable reasoning budgets, and pairs Unsloth Desktop Lemonade/FastFlowLM Q4NX NPU inference.
#!/usr/bin/env bash
# Tinker AI — GMKtec EVO-X5 Pro (192GB / 160GB VRAM) + Unsloth XDNA 2 NPU (Q4NX) & Nemotron 3.5 Super VL 120B (Guide #60)
set -euo pipefail
MODE="evox5_nemotron35_super_vl"
GGUF="./models/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Q4_K_M.gguf"
NPU_MODEL="Qwen3.6-35B-A3B-NPU2"
CTX="65536"
PORT="8080"
NPU_PORT="$((PORT + 1))"
echo "=== [EVO-X5 Pro Audit] Checking 192GB UMA, Linux 7.4 iommu_perfopt & XDNA 2 NPU ==="
free -h
cat /sys/module/amdgpu/parameters/iommu_perfopt 2>/dev/null || echo "[TIP] Upgrade to Linux 7.4+ or add iommu=pt for +18-23% UMA tok/s"
ls -l /dev/accel/accel* 2>/dev/null || echo "[WARN] /dev/accel/accel0 not found; verify amdxdna driver for XDNA 2 NPU"
if [ "$MODE" = "evox5_nemotron35_super_vl" ]; then
echo "=== [Radeon 8065S 160GB UMA] Serving Nemotron 3.5 Super VL 120B-A12B (Mamba-2 Hybrid, ctx=$CTX) ==="
exec llama-server \
-m "$GGUF" -ngl 99 -fa on \
--cache-type-k q8_0 --cache-type-v q8_0 \
-c "$CTX" --host 127.0.0.1 --port "$PORT"
elif [ "$MODE" = "unsloth_xdna2_npu_q4nx" ]; then
echo "=== [Unsloth Desktop / Lemonade + FastFlowLM] Serving $NPU_MODEL (Q4NX) on 55 TOPS XDNA 2 NPU ==="
exec lemonade-server serve \
--backend flm \
--model "$NPU_MODEL" \
--host 127.0.0.1 --port "$NPU_PORT"
elif [ "$MODE" = "dual_igpu_npu_agent" ]; then
echo "=== [Dual-Engine Concurrent Mode] iGPU ($PORT: Nemotron 3.5 VL) + XDNA 2 NPU ($NPU_PORT: $NPU_MODEL Q4NX) ==="
lemonade-server serve --backend flm --model "$NPU_MODEL" --host 127.0.0.1 --port "$NPU_PORT" &
exec llama-server \
-m "$GGUF" -ngl 99 -fa on \
--cache-type-k q8_0 --cache-type-v q8_0 \
-c "$CTX" --host 127.0.0.1 --port "$PORT"
else
echo "Unknown EXEC_MODE=$MODE (expected: evox5_nemotron35_super_vl | unsloth_xdna2_npu_q4nx | dual_igpu_npu_agent)" >&2
exit 1
fi
Customizable Parameter Reference for run-evox5-unsloth-npu-nemotron35.sh
This SH artifact exposes 5 template variables compiled client-side in your browser.
| Template Token | Parameter Description | Verified Default | Example Format |
|---|---|---|---|
{{EXEC_MODE}} | Execution Mode (evox5_nemotron35_super_vl | unsloth_xdna2_npu_q4nx | dual_igpu_npu_agent) | evox5_nemotron35_super_vl | evox5_nemotron35_super_vl |
{{NEMOTRON_GGUF}} | Nemotron 3.5 GGUF Model Path | ./models/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Q4_K_M.gguf | ./models/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Q4_K_M.gguf |
{{NPU_Q4NX_MODEL}} | Unsloth XDNA 2 NPU Q4NX Model Tag | Qwen3.6-35B-A3B-NPU2 | Qwen3.6-35B-A3B-NPU2 |
{{CTX_LEN}} | Context Window (Mamba-2 Hybrid Low KV) | 65536 | 65536 |
{{IGPU_PORT}} | iGPU LLM API Port (NPU uses Port + 1) | 8080 | 8080 |
Verified Environment for "192GB Desktop AI Workstation, Native XDNA 2 NPU & Mamba-2 Hybrid MoE: GMKtec EVO-X5 Pro (Ryzen AI Max+ PRO 495 · 160GB VRAM) + Unsloth Desktop Q4NX NPU Running NVIDIA Nemotron 3.5 Super VL (120B-A12B) & Lightning 30B"
GMKtec EVO-X5 Pro (AMD Ryzen AI Max+ PRO 495 · Radeon 8065S 40 CU · 192GB LPDDR5X-8533 / 273 GB/s / 160GB VRAM · 55 TOPS XDNA 2 NPU · Dual 10GbE)
Nemotron 3.5 weights (OpenMDW-1.1) and Unsloth Desktop Q4NX conversions run entirely on local CPU/iGPU/NPU; once downloaded, bind endpoints to 127.0.0.1 for zero outbound telemetry.
- Unsloth Desktop XDNA 2 NPU inference requires models in the specialized Q4NX binary format (converted locally from GGUF on CPU via Unsloth’s Export utility) and runs through Lemonade + FastFlowLM—standard raw GGUF files cannot execute directly on /dev/accel/accel0 without Q4NX conversion.
- Nemotron-3.5-Super-VL-120B-A12B supports 2-minute MP4 video and RGB images via its C-RADIO v4-H vision encoder, plus 1M context via Mamba-2 state-space layers—verify your llama.cpp / vLLM build includes both the Mamba-2 hybrid cache and C-RADIO mmproj projector before feeding multimodal inputs.
- While GMKtec EVO-X5 Pro allows up to 160GB VRAM allocation from its 192GB LPDDR5X-8533 pool, leaving <32GB for Linux/Windows OS + NPU Q4NX host buffers triggers swap stutter; cap iGPU model + KV budgets at <=135–145GB when running dual iGPU + NPU engines concurrently.
3-Step Verification Checklist After Deploying run-evox5-unsloth-npu-nemotron35.sh
- Raised UMA/GTT allocation on GMKtec EVO-X5 Pro (
192GB) while reserving>=32–40 GBfor OS + NPU Q4NX host buffers so concurrent iGPU + NPU workloads never trigger SSD swap. - Converted or pulled a
Q4NXmodel inside Unsloth Desktop (Lemonade + FastFlowLM) and confirmedhttp://127.0.0.1:8081/v1/chat/completionsexecutes on/dev/accel/accel0without spiking GPU utilization. - Verified
enable_thinking=False(fast mode) vsenable_thinking=True(deep reasoning) onNemotron-3.5-Super-VL-120B-A12Band confirmed C-RADIOmmprojcompatibility before running image/video prompts.