ARTIFACT // CHECK-NVIDIA-VRAM-SPILL.SH (SHELL)
NVIDIA PC Model Loading & PCIe Shared-Memory Spillover Probe (check-nvidia-vram-spill.sh)
Compares physical VRAM usage in nvidia-smi against ollama ps processor split to detect silent CPU/PCIe fallback.
[KEY VERIFIED TAKEAWAY / TL;DR]This reproducible configuration artifact "NVIDIA PC Model Loading & PCIe Shared-Memory Spillover Probe" (check-nvidia-vram-spill.sh, SHELL) is paired with Tinker AI Lab's hardware-verified guide "nvidia-pc-vram-diagnostics". It exposes 2 customizable browser variables (TARGET_MODEL=qwen2.5:14b-instruct-q4_K_M, TEST_CTX=16384) with loopback-first security defaults.
01 // HOMELABLocal AI / HomeLab· Verified: 2026-10-23
Browse All 42 Artifacts →check-nvidia-vram-spill.sh
View Paired Guide →NVIDIA PC Model Loading & PCIe Shared-Memory Spillover Probe
Compares physical VRAM usage in nvidia-smi against ollama ps processor split to detect silent CPU/PCIe fallback.
Customizable Variables (Saved in Browser)
UTF-8 Encoded · Live Client-Side Compilation
#!/usr/bin/env bash
# Tinker AI — NVIDIA PC Model Loading & VRAM Spillover Diagnostic (Guide #2)
set -euo pipefail
MODEL="qwen2.5:14b-instruct-q4_K_M"
CTX="16384"
echo "=== [1/4] NVIDIA Driver, CUDA & Physical VRAM Baseline ==="
nvidia-smi --query-gpu=name,driver_version,memory.total,memory.used,memory.free --format=csv
echo "=== [2/4] Triggering Model Load ($MODEL @ num_ctx=$CTX) ==="
curl -sS http://127.0.0.1:11434/api/generate -d "{
\"model\": \"$MODEL\",
\"prompt\": \"Warmup VRAM allocation check.\",
\"stream\": false,
\"options\": { \"num_ctx\": $CTX }
}" > /dev/null
echo "=== [3/4] Inspecting Layer Offload Split (Look for 100% GPU vs CPU/GPU Split) ==="
ollama ps
echo "=== [4/4] Post-Load Physical VRAM Utilization ==="
nvidia-smi --query-gpu=memory.used,memory.free,utilization.gpu --format=csv
PARAMETER SPECIFICATION // 参数化变量规范说明表
Customizable Parameter Reference for check-nvidia-vram-spill.sh
This SHELL artifact exposes 2 template variables compiled client-side in your browser.
| Template Token | Parameter Description | Verified Default | Example Format |
|---|---|---|---|
{{TARGET_MODEL}} | Model Tag to Load | qwen2.5:14b-instruct-q4_K_M | qwen2.5:14b-instruct-q4_K_M |
{{TEST_CTX}} | Context Length (num_ctx) | 16384 | 16384 |
PAIRED HARDWARE MANIFEST // 配套实测环境与边界记录
Verified Environment for "NVIDIA PC Model Loading & VRAM Diagnostics: Spotting PCIe Shared-Memory Spillover"
Tested Hardware & OS
NVIDIA RTX 4070 12GB VRAM · 32GB DDR5 System RAM · Ubuntu 24.04 & Windows 11 WSL2
Pinned Stack & Model Versions
NVIDIA Driver 560.35 / CUDA 12.6Ollama 0.5.7nvidia-container-toolkit 1.16.2
Offline Boundary (100% Local GPU Execution)
All diagnostics use local `nvidia-smi` and `ollama ps` queries without external dependencies.
Known Operational Limits & Failure Modes
- Modern NVIDIA drivers enable CUDA Sysmem Fallback by default: exceeding physical 12GB VRAM by even 600MB pages tensors over PCIe into DDR5 RAM, slashing speed 8–10x without crashing.
- Running an embedding model (`bge-m3`) alongside a 14B Q4_K_M chat model on a 12GB card leaves less than 1.1GB for KV cache, triggering layer spill at num_ctx > 4096.
DEPLOYMENT CHECKLIST // 配置落地核验步骤
3-Step Verification Checklist After Deploying check-nvidia-vram-spill.sh
- Confirmed `ollama ps` shows `100% GPU` under target `num_ctx` workload.
- Set `OLLAMA_MAX_LOADED_MODELS=1` and `OLLAMA_KV_CACHE_TYPE=q8_0` on 8GB–12GB cards.
- If containerized, verified `docker exec <container> nvidia-smi` outputs the GPU table.