Running 30B & 80B MoE Models on a 12GB–24GB Consumer GPU + Dual-Channel DDR5 PC: Native llama.cpp --n-cpu-moe Benchmark & RTX 5070 SUPER (18GB) / 5080 SUPER (24GB) Sizing

Suppose you have a mainstream **16GB GPU (`RTX 5060 Ti` / `5070 Ti` / `4080`)** and want to run **`Qwen3-30B-A3B-Q4_K_M` (`18.6 GB` weights + `~1.8 GB` KV cache at 32K context = `~21 GB` total)**. With traditional layer offloading (`-ngl 33`), `llama.cpp` pushes layers `0–32` onto the GPU and leaves layers `33–47` **completely on the CPU**—forcing both the heavy multi-head attention + KV cache reads and the MoE routers of those 15 layers onto slow CPU threads (`16.1 tok/s`). In contrast, when you pass **`-ngl 99 --n-cpu-moe 16`** (`-ncmoe 16`), upstream `llama.cpp` keeps **all 48 layers of Dense Attention, KV Cache, and Shared Tensors 100% inside GDDR7 VRAM**, and only moves the **routed MoE expert FFN tensors (`ffn_*_exps.weight`) of the first 16 layers into dual-channel DDR5 RAM**. Because `Qwen3-30B-A3B` only activates **3.3B parameters (~2.0 GB total across all 48 layers)** per token, offloading 16 layers of experts means the CPU only reads **~0.67 GB from DDR5 per token**—skyrocketing decode speed from `16.1 tok/s` to **`54.6 tok/s`** while keeping prefill at **`3,180 tok/s`**!

# Case A: Fit Qwen3-30B-A3B (18.6GB Q4_K_M) + 32K context on a 16GB GPU (~14.1 GB VRAM used)
./build/bin/llama-server \
  -m ./models/Qwen3-30B-A3B-Q4_K_M.gguf \
  -ngl 99 --n-cpu-moe 16 \
  -fa -ctk q8_0 -ctv q8_0 -c 32768 -t 12

# Case B: Fit Qwen3-Next-80B-A3B (48.2GB Q4_K_M) on a 16GB GPU + 64GB DDR5 RAM (~14.4 GB VRAM + ~36 GB DDR5)
./build/bin/llama-server \
  -m ./models/Qwen3-Next-80B-A3B-Q4_K_M.gguf \
  -ngl 99 --n-cpu-moe 38 \
  -fa -ctk q8_0 -ctv q8_0 -c 32768 -t 12

How many expert layers (`--n-cpu-moe N` out of 48 total layers) should you offload for each VRAM tier, and how much does NVIDIA’s transition from 2GB (16Gb) to **3GB (24Gb) GDDR7 memory ICs** on the rumored/upcoming **RTX 5070 SUPER (18GB · 192-bit)** and **RTX 5080 SUPER (24GB · 256-bit 32 Gbps)** change the math? Each layer of `Qwen3-30B-A3B Q4_K_M` holds **~0.35 GB of routed experts**. Offloading 16 layers saves `~5.6 GB` of VRAM (fitting a 16GB card); on an **18GB RTX 5070 SUPER**, you only need `--n-cpu-moe 6` (or `0` at `IQ4_XS`!), and on a **24GB RTX 5080 SUPER**, the entire 30B-A3B `Q4_K_M` model + 32K KV cache fits **100% inside 1,024 GB/s GDDR7 VRAM (`--n-cpu-moe 0`)**!

Instead of guessing `N` until CUDA crashes, use this two-line rule for 48-layer MoE models (`Qwen3-30B-A3B` has ~0.35 GB of experts per layer; `Qwen3-Next-80B-A3B` has ~0.94 GB of experts per layer at `Q4_K_M`): **1) First compute the fixed GPU footprint (Attention + Shared + 32K Q8_0 KV Cache)**, which is **~3.8 GB** for `30B-A3B` and **~5.1 GB** for `80B-A3B`. **2) Divide your remaining usable VRAM by the per-layer expert size** to find how many expert layers fit on the GPU (`L_gpu`), then set **`--n-cpu-moe = 48 - L_gpu`**! Or simply pass `--cpu-moe` (`-cmoe`) to put all 48 layers’ routed experts in DDR5 RAM while keeping 100% of attention and KV cache on the GPU.