Deskside AI Supercomputer vs WRX90 Tower for 671B–1T Local LLMs: NVIDIA DGX Station GB300 (748GB: 252GB HBM3e @ 7.1 TB/s + 496GB LPDDR5X) vs AMD Threadripper PRO 9995WX (Zen 5 · 8-Ch DDR5-6400 + Dual RTX PRO 6000 192GB)

Unlike the compact 128GB **DGX Spark (GB10)** which relies on a single 256-bit LPDDR5X memory pool (`273 GB/s`), the deskside **NVIDIA DGX Station GB300** brings a genuine server-class **Grace Blackwell Ultra Superchip** to a 1,600W liquid-cooled tower: 1. **Tier 1 — 252 GB HBM3e on Blackwell Ultra (`7,100 GB/s`):** This is 8.6× faster than a Mac Studio M3 Ultra (`819 GB/s`) and 3.96× faster than an RTX 5090 / RTX PRO 6000 (`1,792 GB/s`). Any model or active KV cache that fits within **~240 GB usable HBM3e** (including **`Qwen3-235B-A22B` in FP8 (`238 GB`) or Q4_K_M (`142 GB`)** and **30B–120B dense models**) executes with zero PCIe bottleneck. 2. **Tier 2 — 496 GB LPDDR5X on 72-Core Grace CPU (`396 GB/s` over `900 GB/s` NVLink-C2C):** When running **`DeepSeek-R1-671B` (`404 GB` at `Q4_K_M` or `675 GB` at `FP8`)**, the model exceeds the 252 GB HBM3e pool. Thanks to hardware **NVLink-C2C (`900 GB/s` bidirectional, cache-coherent ATS)**, the Blackwell Ultra GPU can directly read routed MoE expert weights stored in the 496 GB Grace LPDDR5X without PCIe DMA copies—or keep all MLA Attention layers + KV cache + shared experts inside the `7.1 TB/s` HBM3e while streaming the `37B` routed experts from LPDDR5X. 3. **AMD Ryzen Threadripper PRO 9995WX ("Shimada Peak" · 96 Zen 5 Cores · WRX90):** For teams that want standard ATX/SSI-EEB modularity, upgradeable GPUs, and x86 Linux compatibility, the 96-core **Threadripper PRO 9995WX** pairs **full-width 512-bit AVX-512 pipelines** (a massive upgrade over Zen 4’s double-pumped 256-bit execution) with **8-channel DDR5-6400 ECC RDIMM (`409.6 GB/s` theoretical, `365 GB/s` measured STREAM triad)** and **128 native PCIe 5.0 lanes**. Slotting **2× RTX PRO 6000 Blackwell (192GB GDDR7 @ `1,792 GB/s` each)** + **512GB 8-channel DDR5** creates a **704 GB hybrid workstation** where both GPUs run at full `PCIe 5.0 ×16` (`64 GB/s` per card).

We benchmarked **`Qwen3-235B-A22B`** (`22B` active per token) and **`DeepSeek-R1-671B`** (`37B` active per token) at a `16,384`-token context window on our **Threadripper PRO 9995WX + 2× RTX PRO 6000 Blackwell (192GB GDDR7 + 512GB 8-Ch DDR5-6400)** workstation, comparing it against lab-measured **Mac Studio M3 Ultra (512GB)**, **AMD EPYC 9354 (12-Ch DDR5 + 32GB GPU)**, and spec-derived estimates (`[Spec-Est.]`) for **NVIDIA DGX Station GB300 (748GB)**:

To extract all **`365 GB/s`** of 8-channel DDR5-6400 bandwidth on **Threadripper PRO 9995WX**, you must configure two settings before launching `llama-server`: first, set **NUMA Nodes Per Socket (`NPS1` or `NPS4` with `--interleave=all`)** in the WRX90 BIOS so all 8 memory controllers feed active CCDs evenly; second, disable Linux Auto-NUMA (`sudo sysctl -w kernel.numa_balancing=0`) so the kernel does not stall token generation migrating 200GB+ of GGUF pages across CCDs. Below is the complete verification and launch workflow.

# 1. Verify all 8 DDR5 RDIMM channels are populated at 6400 MT/s & check Zen 5 AVX-512 flags
sudo dmidecode -t memory | grep -E "Size:|Configured Memory Speed:|Locator:" | grep -v "No Module"
lscpu | grep -E "Model name|NUMA|avx512_vnni|avx512_bf16"

# 2. Check GPU VRAM & NVLink-C2C / PCIe Gen 5 x16 link width
nvidia-smi --query-gpu=index,name,memory.total,pcie.link.gen.current,pcie.link.width.current --format=csv
nvidia-smi nvlink --status 2>/dev/null || true

# 3. Disable Auto-NUMA page thrashing & launch DeepSeek-R1-671B across 2x 96GB GPUs + 8-Ch DDR5
sudo sysctl -w kernel.numa_balancing=0
numactl --interleave=all ./build/bin/llama-server \
  -m /nvme/models/DeepSeek-R1-671B-Q4_K_M.gguf \
  -ngl 99 --n-cpu-moe 35 \
  -fa -ctk q8_0 -ctv q8_0 -c 32768 -t 64 --host 0.0.0.0 --port 8080