New Hardware GPU Memory & Driver Trap (AMD Strix Halo 15.5GB GTT / Intel Arc B580 / DGX Spark UMA)

On Linux kernels 6.14+, the legacy `amdgpu.gttsize` boot parameter is deprecated and ignored on Strix Halo (`gfx1151`), leaving the default TTM/GTT pool capped at ~15.5GB unless configured via BIOS UMA Frame Buffer or `amdttm.pages_limit`.

1. Recommended: Enter BIOS -> Advanced -> UMA Frame Buffer Size and set it to 96GB (leaving 32GB for Linux OS on a 128GB machine) on Kernel 6.16.9+.
2. Or for dynamic GTT allocation without locking static BIOS RAM: replace `amdgpu.gttsize` with `amdttm.pages_limit=25165824 amdttm.page_pool_size=25165824` (96GB in 4KB pages) in `/etc/default/grub` and reboot.

# Set 96GB (25,165,824 x 4KB pages) in GRUB for AMD Strix Halo (gfx1151):
sudo sed -i 's/amdgpu.gttsize=[0-9]*//g' /etc/default/grub
sudo grubby --update-kernel=ALL --args="amdttm.pages_limit=25165824 amdttm.page_pool_size=25165824" || sudo update-grub
# Reboot and verify 96GB pool

[BOUNDARY]Never assign 100% of 128GB RAM to `amdttm.pages_limit`; always leave at least 24GB–32GB for the Linux kernel, Docker daemon, and mmap buffers to avoid OOM-killer freezes.

Default `rocBLAS` fallback routines lack tuned matrix kernels for RDNA 3.5 (`gfx1151`), causing untuned ROCm builds to run slower than Vulkan (`RADV`). Routing GEMM calls through `hipBLASLt` on ROCm 7.2+ unlocks native `gfx1151` acceleration.

1. Export `ROCBLAS_USE_HIPBLASLT=1`, `OLLAMA_FLASH_ATTENTION=1`, and `OLLAMA_KV_CACHE_TYPE=q8_0` in your Ollama systemd service or `llama-server` environment.
2. For maximum single-user stability without ROCm driver dependencies, use the `llama.cpp` Vulkan (`RADV`) backend, which achieves 78+ tok/s on `Qwen3-30B-A3B-Q4_K_M`.

sudo mkdir -p /etc/systemd/system/ollama.service.d
cat << 'EOF' | sudo tee /etc/systemd/system/ollama.service.d/strix-halo.conf
[Service]
Environment="ROCBLAS_USE_HIPBLASLT=1"
Environment="OLLAMA_FLASH_ATTENTION=1"
Environment="OLLAMA_KV_CACHE_TYPE=q8_0"
EOF
sudo systemctl daemon-reload && sudo systemctl restart ollama

[BOUNDARY]Strix Halo has ~256 GB/s LPDDR5X bandwidth: sparse MoE models (`Qwen3-30B-A3B`, 3.3B active) hit 75–88 tok/s, while dense 70B Q4 models are physically bandwidth-bound to ~6 tok/s.

Intel Arc B580 (`Battlemage Xe2`) requires Linux Kernel 6.12+ (`xe` driver) and explicit `/dev/dri` + numeric `render` group GID passthrough in Docker; otherwise `llama.cpp` / IPEX-LLM silently falls back to CPU.

1. Look up the numeric group ID of `/dev/dri/renderD128` via `stat -c "%g" /dev/dri/renderD128` on the host.
2. Pass `--device /dev/dri:/dev/dri --group-add <RENDER_GID>` and `GGML_VK_VISIBLE_DEVICES=0` (or `ONEAPI_DEVICE_SELECTOR=level_zero:0` for IPEX-LLM SYCL) to the container.

RENDER_GID=$(stat -c "%g" /dev/dri/renderD128)
docker run -d --name tinker-arc-llm \
  --device /dev/dri:/dev/dri \
  --group-add "$RENDER_GID" \
  -e GGML_VK_VISIBLE_DEVICES=0 \
  -p 127.0.0.1:11434:11434 \
  ollama/ollama:0.5.7

[BOUNDARY]Intel Arc B580 has 12GB GDDR6 (456 GB/s) with a ~10.6GB safe usable ceiling; keep 8B–12B models under num_ctx=16384 with q8_0 KV cache.

On ARM64 Unified Memory systems (NVIDIA DGX Spark GB10 128GB & Jetson Orin Nano Super 8GB), the Linux kernel caches `.gguf` files in RAM (`buff/cache`). Unlike standard `malloc`, CUDA unified memory (`cudaMallocManaged`) cannot always evict dirty filesystem page caches fast enough during contiguous 70B+ or 4B allocations, triggering spurious CUDA OOMs.

1. Drop clean filesystem page caches (`echo 3 | sudo tee /proc/sys/vm/drop_caches`) before loading a 70B–120B model on DGX Spark or a 4B model on Jetson Orin Nano Super.
2. On Jetson Orin Nano Super (8GB), disable the Ubuntu GNOME desktop (`sudo systemctl set-default multi-user.target`) to permanently reclaim ~1.1GB of unified memory.

# 1. Immediately flush Linux filesystem page cache before loading large models:
sync && echo 3 | sudo tee /proc/sys/vm/drop_caches

# 2. Tune vm.vfs_cache_pressure so unified memory prefers GPU allocations over file cache:
sudo sysctl -w vm.vfs_cache_pressure=200

[BOUNDARY]Dropping `drop_caches=3` is non-destructive (dirty pages are synced first) and typically frees 15GB–45GB on DGX Spark and 0.8GB–1.5GB on Jetson.