TinkerLab
01 // HOMELABNVIDIA GPUDocker / ComposeStreaming / API~15 min read & lab

192GB Desktop AI Workstation, Native XDNA 2 NPU & Mamba-2 Hybrid MoE: GMKtec EVO-X5 Pro (Ryzen AI Max+ PRO 495 · 160GB VRAM) + Unsloth Desktop Q4NX NPU Running NVIDIA Nemotron 3.5 Super VL (120B-A12B) & Lightning 30B

End-to-end engineering blueprint combining three November 2026 client-AI milestones: the GMKtec EVO-X5 Pro 192GB unified-memory mini workstation (Ryzen AI Max+ PRO 495, 273 GB/s LPDDR5X-8533, 160GB GPU VRAM ceiling, 55 TOPS XDNA 2 NPU, Dual 10GbE & 24TB M.2), AMD + Unsloth Desktop native XDNA 2 NPU support (Lemonade + FastFlowLM + Q4NX GGUF converter serving 100+ agent tools), and NVIDIA’s newly open-sourced Nemotron 3.5 Super VL (120B-A12B Mamba-2 + Transformer MoE + C-RADIO v4-H vision/video, 1M context, native MTP, OpenMDW-1.1) & Nemotron 3.5 Lightning (30B-A3B).

[SPEC-DERIVED TAKEAWAY / TL;DR][Evidence level: spec-derived estimate, not lab-measured] "192GB Desktop AI Workstation, Native XDNA 2 NPU & Mamba-2 Hybrid MoE: GMKtec EVO-X5 Pro (Ryzen AI Max+ PRO 495 · 160GB VRAM) + Unsloth Desktop Q4NX NPU Running NVIDIA Nemotron 3.5 Super VL (120B-A12B) & Lightning 30B" (spec basis: GMKtec EVO-X5 Pro (AMD Ryzen AI Max+ PRO 495 · Radeon 8065S 40 CU · 192GB LPDDR5X-8533 / 273 GB/s / 160GB VRAM · 55 TOPS XDNA 2 NPU · Dual 10GbE); suggested stack: Unsloth Desktop (Nov 2026 · AMD Lemonade SDK + FastFlowLM Q4NX NPU), llama.cpp v1.0.0+ / ROCm 7.2+ / Linux 7.4+ (amdgpu.iommu_perfopt=1), nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B (NVFP4 / GGUF · OpenMDW-1.1), nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B & Qwen3.6-35B-A3B-NPU2 (Q4NX); specs checked 2026-11-20): End-to-end engineering blueprint combining three November 2026 client-AI milestones: the GMKtec EVO-X5 Pro 192GB unified-memory mini workstation (Ryzen AI Max+ PRO 495, 273 GB/s LPDDR5X-8533, 160GB GPU VRAM ceiling, 55 TOPS XDNA 2 NPU, Dual 10GbE & 24TB M.2), AMD + Unsloth Desktop native XDNA 2 NPU support (Lemonade + FastFlowLM + Q4NX GGUF converter serving 100+ agent tools), and NVIDIA’s newly open-sourced Nemotron 3.5 Super VL (120B-A12B Mamba-2 + Transformer MoE + C-RADIO v4-H vision/video, 1M context, native MTP, OpenMDW-1.1) & Nemotron 3.5 Lightning (30B-A3B). Offline capability: "100% Local Desktop Multimodal Reasoning + NPU Agent Execution" (Nemotron 3.5 weights (OpenMDW-1.1) and Unsloth Desktop Q4NX conversions run entirely on local CPU/iGPU/NPU; once downloaded, bind endpoints to 127.0.0.1 for zero outbound telemetry.).
REPRODUCIBILITY MANIFEST // SPEC ESTIMATE & PROTOCOL
Spec estimate · not lab-measured · specs checked 2026-11-20
Spec Basis Hardware (Not Lab-Measured)

GMKtec EVO-X5 Pro (AMD Ryzen AI Max+ PRO 495 · Radeon 8065S 40 CU · 192GB LPDDR5X-8533 / 273 GB/s / 160GB VRAM · 55 TOPS XDNA 2 NPU · Dual 10GbE)

Pinned Stack & Model Versions
Unsloth Desktop (Nov 2026 · AMD Lemonade SDK + FastFlowLM Q4NX NPU)llama.cpp v1.0.0+ / ROCm 7.2+ / Linux 7.4+ (amdgpu.iommu_perfopt=1)nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B (NVFP4 / GGUF · OpenMDW-1.1)nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B & Qwen3.6-35B-A3B-NPU2 (Q4NX)
Offline & Network Boundary
100% Local Desktop Multimodal Reasoning + NPU Agent Execution

Nemotron 3.5 weights (OpenMDW-1.1) and Unsloth Desktop Q4NX conversions run entirely on local CPU/iGPU/NPU; once downloaded, bind endpoints to 127.0.0.1 for zero outbound telemetry.

Known Limits & Failure Modes
  • Unsloth Desktop XDNA 2 NPU inference requires models in the specialized Q4NX binary format (converted locally from GGUF on CPU via Unsloth’s Export utility) and runs through Lemonade + FastFlowLM—standard raw GGUF files cannot execute directly on /dev/accel/accel0 without Q4NX conversion.
  • Nemotron-3.5-Super-VL-120B-A12B supports 2-minute MP4 video and RGB images via its C-RADIO v4-H vision encoder, plus 1M context via Mamba-2 state-space layers—verify your llama.cpp / vLLM build includes both the Mamba-2 hybrid cache and C-RADIO mmproj projector before feeding multimodal inputs.
  • While GMKtec EVO-X5 Pro allows up to 160GB VRAM allocation from its 192GB LPDDR5X-8533 pool, leaving <32GB for Linux/Windows OS + NPU Q4NX host buffers triggers swap stutter; cap iGPU model + KV budgets at <=135–145GB when running dual iGPU + NPU engines concurrently.
Hit an edge-case failure?Open Troubleshooter →

1. NVIDIA Nemotron 3.5 Super VL (120B-A12B) & Lightning (30B-A3B): Why Mamba-2 + MoE Loves 192GB Unified Memory

Released under NVIDIA’s permissive OpenMDW-1.1 license alongside SC26 Nemotron for Science, the Nemotron 3.5 lineup solves the two biggest pain points of desktop unified-memory machines (273 GB/s bandwidth ceiling & long-context KV explosion): - Nemotron-3.5-Super-VL-120B-A12B: Combines Mamba-2 state-space layers interleaved with sparse Transformer Attention and 512 routed MoE experts (120B total, only 12B active per token), paired with the C-RADIO v4-H vision encoder (text + RGB images + up to 2-minute MP4 video), 1M context, native Multi-Token Prediction (MTP) speculative heads, and runtime reasoning toggles (enable_thinking=True/False or off / medium / on). At Q4_K_M (~75 GB) or Q8_0 (~131 GB), it fits completely inside GMKtec EVO-X5 Pro’s 160GB allocatable VRAM with negligible Mamba-2 KV growth at 64K–128K context. - Nemotron-3.5-Lightning-30B-A3B: Only 3B active parameters (~18.7 GB at Q4_K_M) with native MTP—flying past 50–70 tok/s on 273 GB/s unified memory or 24GB discrete GPUs for always-on coding/tool agents.

2. GMKtec EVO-X5 Pro (192GB) + Unsloth Desktop Native XDNA 2 NPU (Q4NX): Dual-Engine Sizing & Benchmarks

In November 2026, AMD and Unsloth made AMD XDNA 2 NPUs (55 TOPS on GMKtec EVO-X5 Pro’s Ryzen AI Max+ PRO 495) the first NPU family natively supported by Unsloth Desktop. Under the hood, Unsloth Desktop manages AMD’s Lemonade server + FastFlowLM (FLM) runtime, includes a built-in CPU GGUF-to-Q4NX converter, and exposes a standard OpenAI-compatible endpoint usable immediately by 100+ agent apps (Open WebUI, Cline, Hermes Agent, OpenClaw, VS Code). This lets a single 192GB EVO-X5 Pro split workloads cleanly across two isolated silicon engines ([Spec-Est.] marks spec-derived numbers):
Table 1: Dual-Engine Execution on GMKtec EVO-X5 Pro (192GB LPDDR5X-8533 · 273 GB/s · Dual 10GbE) vs 24GB–96GB GPUs
Hardware & Execution EngineModel & Format (Memory Footprint)Decode Speed & Context HeadroomBest Local Role
GMKtec EVO-X5 Pro — Radeon 8065S (160GB UMA) (Linux 7.4 PerfOpt)Nemotron-3.5-Super-VL-120B-A12B Q4_K_M (~78 GB) or Q8_0 (~134 GB)~22–31 tok/s (Q4 + MTP [Spec-Est.]) · 64K–128K Mamba-2 ctxDeep multimodal reasoning, PDF/video analysis & science/code synthesis
GMKtec EVO-X5 Pro — XDNA 2 NPU (55 TOPS) (Unsloth Desktop FLM)Qwen3.6-35B-A3B-NPU2 / 4B–9B Q4NX (~6–22 GB host RAM)~18–35 tok/s (<15W NPU power [Spec-Est.]) · zero iGPU contention24/7 background Cline/OpenClaw router, transcription & privacy filter
GMKtec EVO-X5 Pro — Radeon 8065S (160GB UMA)Nemotron-3.5-Lightning-30B-A3B Q4_K_M (~20 GB)~55–75 tok/s (3B active + MTP [Spec-Est.]) · 128K ctx effortlessUltra-responsive interactive coding & multi-agent tool loops
NVIDIA RTX PRO 6000 Blackwell (96GB GDDR7 · 1.8 TB/s)Nemotron-3.5-Super-VL-120B-A12B-NVFP4 (~68 GB native FP4)~110–150+ tok/s (NVFP4 + MTP [Spec-Est.]) · 128K ctxHigh-concurrency team server (higher cost & 300–600W TDP)

3. Reproducible Setup: Serve Nemotron 3.5 Super VL on 160GB UMA & Connect Unsloth XDNA 2 NPU (Q4NX)

Verify 192GB memory + Linux 7.4 amdgpu.iommu_perfopt=1 and /dev/accel/accel0 first, then run Nemotron-3.5-Super-VL-120B-A12B on port 8080 (Radeon 8065S) alongside Unsloth Desktop’s Lemonade + FastFlowLM (Q4NX) OpenAI endpoint on port 8081 (XDNA 2 NPU), controlling reasoning depth per request with enable_thinking.
Launch Dual-Engine Nemotron 3.5 Super VL (iGPU) + Unsloth Lemonade Q4NX (XDNA 2 NPU) on EVO-X5 Pro
# 1) Verify 192GB RAM, Linux 7.4 AMD IOMMU PerfOpt, and /dev/accel/accel0 XDNA 2 NPU
free -h && ls -l /dev/accel/accel* /dev/kfd
cat /sys/module/amdgpu/parameters/iommu_perfopt 2>/dev/null || echo "Use Linux 7.4+ or iommu=pt"

# 2) Start XDNA 2 NPU OpenAI endpoint via Lemonade + FastFlowLM (Q4NX exported by Unsloth Desktop)
lemonade-server serve --backend flm --model Qwen3.6-35B-A3B-NPU2 --host 127.0.0.1 --port 8081 &

# 3) Serve NVIDIA Nemotron-3.5-Super-VL-120B-A12B on Radeon 8065S 160GB UMA (64K Mamba-2 ctx)
llama-server -m ./models/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Q4_K_M.gguf \
  -ngl 99 -fa on --cache-type-k q8_0 --cache-type-v q8_0 -c 65536 --host 127.0.0.1 --port 8080
FIELD Q&A // 常见实测疑问与边界解答

Frequently Asked Questions & Edge Cases

Q: How does GMKtec EVO-X5 Pro (192GB) compare to a 128GB Strix Halo box like Minisforum MS-S1 Max for Nemotron 3.5 Super VL 120B?

On a 128GB box (~96–108GB usable UMA), Nemotron-3.5-Super-VL-120B at Q4_K_M (~78 GB) leaves only ~20–25 GB for OS, browser, Docker RAG, and an NPU Q4NX model—making Q8_0 (~134 GB) impossible. GMKtec EVO-X5 Pro’s 192GB LPDDR5X-8533 (273 GB/s, +6.6% higher bandwidth + +64GB RAM) lets you either run near-lossless Q8_0 (~134 GB) comfortably, or run Q4_K_M (~78 GB) on the Radeon 8065S simultaneously with a 22 GB Q4NX MoE model on the XDNA 2 NPU plus Dual 10GbE NAS indexing with zero memory pressure.

Q: Why does Unsloth Desktop convert GGUF to Q4NX instead of running GGUF directly on AMD XDNA 2 NPUs?

Standard GGUF super-block layouts (Q4_K_M) are tuned for GPU SIMT warps and CPU AVX-512/NEON vector lanes. AMD’s XDNA 2 is a spatial dataflow AIE (AI Engine) tile array with fixed SRAM scratchpads per tile. FastFlowLM’s Q4NX format pre-packs quantized weights, scales, and tiling alignments to stream cleanly through XDNA 2 AIE tiles without CPU dequantization stalls—delivering <15W sustained token generation while leaving the CPU and Radeon iGPU 100% idle.

REPRODUCIBLE ARTIFACT // 配套可参数化附件

GMKtec EVO-X5 Pro (192GB / 160GB UMA) + Unsloth XDNA 2 NPU (Q4NX) & NVIDIA Nemotron 3.5 Super VL 120B Runner

Audits 160GB Radeon 8065S UMA & /dev/accel/accel0 XDNA 2 NPU on GMKtec EVO-X5 Pro (Ryzen AI Max+ PRO 495), runs NVIDIA Nemotron-3.5-Super-VL-120B-A12B or Lightning 30B with configurable reasoning budgets, and pairs Unsloth Desktop Lemonade/FastFlowLM Q4NX NPU inference.

run-evox5-unsloth-npu-nemotron35.sh ↗
Edit variables above to update the configuration live
#!/usr/bin/env bash
# Tinker AI — GMKtec EVO-X5 Pro (192GB / 160GB VRAM) + Unsloth XDNA 2 NPU (Q4NX) & Nemotron 3.5 Super VL 120B (Guide #60)
set -euo pipefail

MODE="evox5_nemotron35_super_vl"
GGUF="./models/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Q4_K_M.gguf"
NPU_MODEL="Qwen3.6-35B-A3B-NPU2"
CTX="65536"
PORT="8080"
NPU_PORT="$((PORT + 1))"

echo "=== [EVO-X5 Pro Audit] Checking 192GB UMA, Linux 7.4 iommu_perfopt & XDNA 2 NPU ==="
free -h
cat /sys/module/amdgpu/parameters/iommu_perfopt 2>/dev/null || echo "[TIP] Upgrade to Linux 7.4+ or add iommu=pt for +18-23% UMA tok/s"
ls -l /dev/accel/accel* 2>/dev/null || echo "[WARN] /dev/accel/accel0 not found; verify amdxdna driver for XDNA 2 NPU"

if [ "$MODE" = "evox5_nemotron35_super_vl" ]; then
  echo "=== [Radeon 8065S 160GB UMA] Serving Nemotron 3.5 Super VL 120B-A12B (Mamba-2 Hybrid, ctx=$CTX) ==="
  exec llama-server \
    -m "$GGUF" -ngl 99 -fa on \
    --cache-type-k q8_0 --cache-type-v q8_0 \
    -c "$CTX" --host 127.0.0.1 --port "$PORT"
elif [ "$MODE" = "unsloth_xdna2_npu_q4nx" ]; then
  echo "=== [Unsloth Desktop / Lemonade + FastFlowLM] Serving $NPU_MODEL (Q4NX) on 55 TOPS XDNA 2 NPU ==="
  exec lemonade-server serve \
    --backend flm \
    --model "$NPU_MODEL" \
    --host 127.0.0.1 --port "$NPU_PORT"
elif [ "$MODE" = "dual_igpu_npu_agent" ]; then
  echo "=== [Dual-Engine Concurrent Mode] iGPU ($PORT: Nemotron 3.5 VL) + XDNA 2 NPU ($NPU_PORT: $NPU_MODEL Q4NX) ==="
  lemonade-server serve --backend flm --model "$NPU_MODEL" --host 127.0.0.1 --port "$NPU_PORT" &
  exec llama-server \
    -m "$GGUF" -ngl 99 -fa on \
    --cache-type-k q8_0 --cache-type-v q8_0 \
    -c "$CTX" --host 127.0.0.1 --port "$PORT"
else
  echo "Unknown EXEC_MODE=$MODE (expected: evox5_nemotron35_super_vl | unsloth_xdna2_npu_q4nx | dual_igpu_npu_agent)" >&2
  exit 1
fi

More Reproducible Engineering Guides

View All 60 Guides →