ARTIFACT // RUN-BLACKWELL-SM120-LLM.SH (SHELL)
NVIDIA Blackwell (RTX 5090 / 5060 Ti `sm_120`) Power-Cap & CUDA 12.8 Launcher (run-blackwell-sm120-llm.sh)
Applies persistent nvidia-smi power limit (e.g. 450W on RTX 5090 or 165W on RTX 5060 Ti 16GB), verifies Compute Capability 12.0 (sm_120), and runs llama-server with MMQ + FlashAttention.
[KEY VERIFIED TAKEAWAY / TL;DR]This reproducible configuration artifact "NVIDIA Blackwell (RTX 5090 / 5060 Ti `sm_120`) Power-Cap & CUDA 12.8 Launcher" (run-blackwell-sm120-llm.sh, SHELL) is paired with Tinker AI Lab's hardware-verified guide "rtx-5090-vs-rtx-5060ti-16gb-blackwell-fp4-sm120-local-llm-benchmark". It exposes 3 customizable browser variables (GPU_POWER_LIMIT_W=450, GGUF_MODEL_PATH=/models/Qwen3-30B-A3B-Q4_K_M.gguf, CTX_SIZE=32768) with loopback-first security defaults.
01 // HOMELABLocal AI / HomeLab· Verified: 2026-10-28
Browse All 42 Artifacts →run-blackwell-sm120-llm.sh
View Paired Guide →NVIDIA Blackwell (RTX 5090 / 5060 Ti `sm_120`) Power-Cap & CUDA 12.8 Launcher
Applies persistent nvidia-smi power limit (e.g. 450W on RTX 5090 or 165W on RTX 5060 Ti 16GB), verifies Compute Capability 12.0 (sm_120), and runs llama-server with MMQ + FlashAttention.
Customizable Variables (Saved in Browser)
UTF-8 Encoded · Live Client-Side Compilation
#!/usr/bin/env bash # Tinker AI — NVIDIA Blackwell (sm_120) Power Tuning & MMQ Inference Runner (Guide #26) set -euo pipefail PL_WATTS="450" MODEL_PATH="/models/Qwen3-30B-A3B-Q4_K_M.gguf" CTX="32768" echo "=== [1/3] Checking NVIDIA Blackwell Compute Capability (Expect 12.0 / sm_120) ===" nvidia-smi --query-gpu=index,name,driver_version,compute_cap,power.draw,power.limit --format=csv echo "=== [2/3] Locking Persistent GPU Power Cap to $PL_WATTS W ===" sudo nvidia-smi -pm 1 sudo nvidia-smi -pl "$PL_WATTS" echo "=== [3/3] Launching llama-server (Native sm_120 MMQ + FlashAttention + Q8_0 KV) ===" ./build/bin/llama-server \ -m "$MODEL_PATH" \ -c "$CTX" \ -ngl 999 \ -fa \ -ctk q8_0 -ctv q8_0 \ --host 127.0.0.1 \ --port 11434
PARAMETER SPECIFICATION // 参数化变量规范说明表
Customizable Parameter Reference for run-blackwell-sm120-llm.sh
This SHELL artifact exposes 3 template variables compiled client-side in your browser.
| Template Token | Parameter Description | Verified Default | Example Format |
|---|---|---|---|
{{GPU_POWER_LIMIT_W}} | GPU Power Limit in Watts (nvidia-smi -pl) | 450 | 450 (RTX 5090) or 165 (RTX 5060 Ti 16GB) |
{{GGUF_MODEL_PATH}} | Path to GGUF / MXFP4 Model File | /models/Qwen3-30B-A3B-Q4_K_M.gguf | /models/Qwen3-30B-A3B-Q4_K_M.gguf |
{{CTX_SIZE}} | Context Window Size (num_ctx) | 32768 | 32768 |
PAIRED HARDWARE MANIFEST // 配套实测环境与边界记录
Verified Environment for "NVIDIA Blackwell Local LLM Benchmark: RTX 5090 (32GB) vs RTX 5060 Ti (16GB), Native FP4/MXFP4, CUDA 12.8 sm_120 & 450W Power Tuning"
Tested Hardware & OS
NVIDIA GeForce RTX 5090 (32GB GDDR7 · 1,792 GB/s) & RTX 5060 Ti (16GB GDDR7 · 448 GB/s) · AMD Ryzen 9 9950X · 64GB DDR5 · Ubuntu 24.04.2 LTS
Pinned Stack & Model Versions
NVIDIA Driver 570.133.07 + CUDA 12.8 (sm_120)llama.cpp b4910 (-DCMAKE_CUDA_ARCHITECTURES=120)vLLM 0.8.2 (PyTorch 2.7.0+cu128)Qwen3-14B / Qwen3-30B-A3B / Qwen2.5-Coder-32B
Offline Boundary (100% Air-Gapped Capable)
All CUDA 12.8 kernels, power-cap commands, and GGUF/FP4 models execute locally without cloud telemetry.
Known Operational Limits & Failure Modes
- RTX 5060 Ti 16GB has a 128-bit bus (448 GB/s GDDR7): it fits 14B models with 32K context effortlessly, but dense 32B Q4_K_M (~19.8GB) spills over PCIe and drops from 39 tok/s to 4.8 tok/s.
- Compiling `llama.cpp` with `-DGGML_CUDA_FORCE_CUBLAS=ON` on `sm_120` disables quantized MMQ kernels and slashes token speed by up to 78%.
DEPLOYMENT CHECKLIST // 配置落地核验步骤
3-Step Verification Checklist After Deploying run-blackwell-sm120-llm.sh
- Verified NVIDIA Driver >= 570.124 and CUDA 12.8 (`compute_cap = 12.0` / `sm_120`) before launching containers.
- Applied persistent power limit (`nvidia-smi -pm 1 && nvidia-smi -pl 450` on RTX 5090 or `165` on RTX 5060 Ti).
- Confirmed `GGML_CUDA_FORCE_CUBLAS` is OFF so `llama.cpp` uses native quantized MMQ kernels on `sm_120`.