EPYC SP7 (16-Ch MRDIMM) & Mac Studio M5 Ultra Frontier MoE Launcher (DeepSeek-V4.1-Flash / GLM-5.3 / Muse Spark) (launch-sp7-m5-ultra-frontier-moe.sh)
Audits 16-channel SP7 RDIMM/MRDIMM population and disables auto-NUMA on Linux, or raises Metal wired limit to 480GB on 512GB Mac Studio M5 Ultra before launching 552B–816B MoE inference.
EPYC SP7 (16-Ch MRDIMM) & Mac Studio M5 Ultra Frontier MoE Launcher (DeepSeek-V4.1-Flash / GLM-5.3 / Muse Spark)
Audits 16-channel SP7 RDIMM/MRDIMM population and disables auto-NUMA on Linux, or raises Metal wired limit to 480GB on 512GB Mac Studio M5 Ultra before launching 552B–816B MoE inference.
#!/usr/bin/env bash # Tinker AI — AMD EPYC SP7 (16-Ch MRDIMM) & Mac Studio M5 Ultra Frontier MoE Launcher (Guide #53) set -euo pipefail TARGET="epyc_sp7" MODEL="/nvme/models/DeepSeek-V4.1-Flash-Q4_K_M.gguf" CTX="131072" PORT="8080" if [ "$TARGET" = "epyc_sp7" ]; then echo "=== [EPYC SP7 16-Ch Mode] Auditing DDR5/MRDIMM Speed & Disabling Auto-NUMA ===" sudo dmidecode -t memory | grep -E "Speed:|Configured Memory Speed:" | sort -u || true sudo sysctl -w kernel.numa_balancing=0 exec numactl --interleave=all llama-server -m "$MODEL" -t 96 -fa on --cache-type-k fp8_e4m3 -c "$CTX" --port "$PORT" elif [ "$TARGET" = "mac_m5_ultra" ]; then echo "=== [Mac Studio M5 Ultra 512GB Mode] Unlocking 480GB Metal Wired Pool ===" sudo sysctl -w iogpu.wired_limit_mb=491520 exec mlx_lm.server --model "$MODEL" --port "$PORT" else echo "Unknown HOST_TARGET=$TARGET (expected: epyc_sp7 | mac_m5_ultra)" >&2 exit 1 fi
Customizable Parameter Reference for launch-sp7-m5-ultra-frontier-moe.sh
This SH artifact exposes 4 template variables compiled client-side in your browser.
| Template Token | Parameter Description | Verified Default | Example Format |
|---|---|---|---|
{{HOST_TARGET}} | Host Mode (epyc_sp7 | mac_m5_ultra) | epyc_sp7 | epyc_sp7 |
{{MODEL_PATH}} | GGUF or MLX Model Path / Repo | /nvme/models/DeepSeek-V4.1-Flash-Q4_K_M.gguf | /nvme/models/DeepSeek-V4.1-Flash-Q4_K_M.gguf |
{{CTX_TOKENS}} | Context Window (-c) | 131072 | 131072 |
{{SERVER_PORT}} | OpenAI API Port | 8080 | 8080 |
Verified Environment for "AMD 6th Gen EPYC "Venice" SP7 (16-Ch MRDIMM · 1.6 TB/s) & Helios vs Mac Studio M5 Ultra (512GB): Running DeepSeek-V4.1-Flash (552B), GLM-5.3 (744B) & Muse Spark Open (816B)"
AMD 6th Gen EPYC "Venice" SP7 (16-Ch DDR5-8000 / MRDIMM-12800, 1TB RAM) & Helios Open Rack Blueprint vs Apple Mac Studio M5 Ultra (512GB Unified @ 1.2 TB/s)
DeepSeek-V4.1-Flash uses the permissive MIT license; GLM-5.3 and Meta Muse Spark Open use custom permissive licenses with commercial revenue/MaaS review thresholds—verify compliance before hosting public APIs.
- Populating 16-channel SP7 with JEDEC MRDIMM-12800 (1.6 TB/s/socket) requires 1 DPC (16 DIMMs/socket) and compatible SP7 motherboards; mixing RDIMM speeds downclocks the entire memory controller.
- Meta Muse Spark Open (816B total) weighs ~470–495 GB in 4-bit Q4_K_M—exceeding the ~475GB safe Metal wired pool on a 512GB Mac Studio M5 Ultra at long contexts unless quantized to 3.5-bit/IQ3_XS or run on 1TB+ SP7 EPYC.
- Mac Studio M5 Ultra runs whisper-quiet under ~220W for 1–4 concurrent users, whereas an EPYC SP7 / Helios node draws 500W–1,200W+ and requires server rack airflow.
3-Step Verification Checklist After Deploying launch-sp7-m5-ultra-frontier-moe.sh
- Verified 16 populated DIMMs (1 DPC) per SP7 socket at rated 8,000 MT/s RDIMM or 12,800 MT/s MRDIMM speed via
dmidecode. - Disabled
kernel.numa_balancing=0and launchedllama-serverwithnumactl --interleave=allto prevent cross-CCD page thrashing. - Raised
iogpu.wired_limit_mb=491520on 512GB Mac Studio M5 Ultra and verified MIT vs custom MaaS license compliance for DeepSeek-V4.1-Flash / GLM-5.3 / Muse Spark Open.