DGX Spark 4-Node Switchless 200GbE Ring (512GB) + LMCache NVMe GDS Launcher (launch-dgx-spark-4node-ring-lmcache.sh)
Audits dual-port ConnectX-7 200GbE ring MTU 9000 & IP forwarding, flushes GB10 UMA page cache, locks NCCL_ALGO=Ring to prevent non-adjacent deadlocks, and starts vLLM TP=4 with LMCache GPUDirect Storage (cuFile) NVMe KV offloading.
DGX Spark 4-Node Switchless 200GbE Ring (512GB) + LMCache NVMe GDS Launcher
Audits dual-port ConnectX-7 200GbE ring MTU 9000 & IP forwarding, flushes GB10 UMA page cache, locks NCCL_ALGO=Ring to prevent non-adjacent deadlocks, and starts vLLM TP=4 with LMCache GPUDirect Storage (cuFile) NVMe KV offloading.
#!/usr/bin/env bash
# Tinker AI — SC26 DGX Spark 4-Node Switchless 200GbE Ring + LMCache GDS Launcher (Guide #55)
set -euo pipefail
IFACES="enP7s7f0,enP7s7f1"
HEAD_IP="10.10.12.1"
MODEL="deepseek-ai/DeepSeek-V4.1-Flash-NVFP4"
KV_DIR="/nvme/lmcache_gds"
MAX_GB="512"
echo "=== [1/4] Auditing ConnectX-7 200GbE Ring MTU 9000, Diagonal IP Forwarding & GDS ==="
IFS=',' read -ra DEV_ARR <<< "$IFACES"
for dev in "${DEV_ARR[@]}"; do
ip -br link show "$dev" || echo "WARN: Ring interface $dev not found"
done
sudo sysctl -w net.ipv4.ip_forward=1 >/dev/null
lsmod | grep -q nvidia_fs && echo "OK: nvidia_fs (GPUDirect Storage cuFile) kernel module loaded" || echo "INFO: nvidia_fs not loaded; LMCache will fall back to POSIX O_DIRECT"
echo "=== [2/4] Flushing GB10 Coherent UMA Page Cache Before Sharded Weight mmap ==="
sync && echo 3 | sudo tee /proc/sys/vm/drop_caches >/dev/null
mkdir -p "$KV_DIR"
echo "=== [3/4] Locking Switchless 4-Node Ring NCCL & LMCache GDS Environment ==="
export NCCL_SOCKET_IFNAME="$IFACES"
export UCX_NET_DEVICES="${DEV_ARR[0]}"
export NCCL_ALGO=Ring
export NCCL_CROSS_NIC=1
export NCCL_BUFFSIZE=4194304
export LMCACHE_USE_EXPERIMENTAL=True
export LMCACHE_LOCAL_CPU=True
export LMCACHE_MAX_LOCAL_CPU_SIZE=16.0
export LMCACHE_CUFILE_BUFFER_SIZE="4096"
export LMCACHE_GDS_PATH="$KV_DIR"
export LMCACHE_MAX_LOCAL_DISK_SIZE="$MAX_GB"
echo "=== [4/4] Starting vLLM TP=4 Distributed Server over Ray ($HEAD_IP:6379) ==="
exec vllm serve "$MODEL" \
--tensor-parallel-size 4 \
--distributed-executor-backend ray \
--quantization nvfp4 \
--max-model-len 131072 \
--kv-transfer-config '{"kv_connector":"LMCacheConnectorV1","kv_role":"kv_both"}' \
--host 0.0.0.0 --port 8000
Customizable Parameter Reference for launch-dgx-spark-4node-ring-lmcache.sh
This SH artifact exposes 5 template variables compiled client-side in your browser.
| Template Token | Parameter Description | Verified Default | Example Format |
|---|---|---|---|
{{RING_IFACES}} | ConnectX-7 Dual-Port Ring Interfaces | enP7s7f0,enP7s7f1 | enP7s7f0,enP7s7f1 |
{{RAY_HEAD_RING_IP}} | Node 1 Ring Head IP (/30 Subnet) | 10.10.12.1 | 10.10.12.1 |
{{MODEL_REPO}} | NVFP4 / MoE Model ID (TP=4 across 512GB) | deepseek-ai/DeepSeek-V4.1-Flash-NVFP4 | deepseek-ai/DeepSeek-V4.1-Flash-NVFP4 |
{{NVME_LMCACHE_DIR}} | PCIe Gen5/Gen6 NVMe GDS KV-Cache Path | /nvme/lmcache_gds | /nvme/lmcache_gds |
{{MAX_DISK_GB}} | Max Local NVMe KV Tier Quota (GB) | 512 | 512 |
Verified Environment for "SC26 Desktop Cluster & NVMe KV Offload: NVIDIA DGX Spark 4-Node Switchless 200GbE Ring (512GB) + LMCache GPUDirect Storage Running Mistral Large 4 (1.05T) & 700B NVFP4"
4x NVIDIA DGX Spark HPC Developer Kit (4x GB10 Grace Blackwell · 512GB Coherent LPDDR5X · 20TB NVMe · Dual-Port ConnectX-7 200GbE Switchless Ring)
Mistral Large 4 uses the Mistral Ecosystem Open-Weight License (v26.10), which permits on-premise commercial deployment and fine-tuning while restricting direct hosted MaaS resale without a commercial agreement.
- In a 4-node switchless ring (Node 1 <-> Node 2 <-> Node 3 <-> Node 4 <-> Node 1), diagonally opposite nodes have no direct cable; leaving NCCL on default Tree mode or omitting /30 static routes causes all_reduce hangs or silent fallback to 1GbE/10GbE LAN.
- Mistral Large 4 (~1.05T total, ~50B active + 1.6B vision encoder) exceeds 512GB in 4-bit (~600+ GB with KV), requiring 3.1-bit IQ3_XXS across the 4-node 512GB ring or a 1TB+ AMD EPYC SP7 server for full 4-bit Q4_K_M.
- GPUDirect Storage (cuFile / nvidia_fs) zero-copy NVMe KV offloading requires raw/ext4/XFS mounts without OS encryption layers and IOMMU passthrough compatibility.
3-Step Verification Checklist After Deploying launch-dgx-spark-4node-ring-lmcache.sh
- Wired all 4 dual-port ConnectX-7 NICs in a closed loop (
P1 -> next P0), setMTU 9000, and configurednet.ipv4.ip_forward=1plus/30diagonal static routes. - Exported
NCCL_ALGO=RingandNCCL_CROSS_NIC=1(preventing switchlessTreehangs) and flushed/proc/sys/vm/drop_cacheson all 4 GB10 nodes before weight loading. - Verified
nvidia_fs(cuFileGDS) orO_DIRECTNVMe path underLMCACHE_GDS_PATHwith an explicit disk quota to accelerate multi-turn 128K TTFT.