Qwen3.8-27B / Flash-Next (51B N-Gram Host Cache) / Muse Glimmer 30B & Ascend 96GB Runner (run-qwen38-glimmer-or-ascend-station.sh)
Runs Qwen3.8-27B (16 KV layers) or Muse Glimmer 30B (2K SWA + global attn) at 128K context on 24GB–32GB GPUs, offloads Qwen3.8-Flash-Next N-gram/MoE tables to DDR5 RAM, or audits Orange Pi AI Station (Ascend 310 96GB CANN) health.
Qwen3.8-27B / Flash-Next (51B N-Gram Host Cache) / Muse Glimmer 30B & Ascend 96GB Runner
Runs Qwen3.8-27B (16 KV layers) or Muse Glimmer 30B (2K SWA + global attn) at 128K context on 24GB–32GB GPUs, offloads Qwen3.8-Flash-Next N-gram/MoE tables to DDR5 RAM, or audits Orange Pi AI Station (Ascend 310 96GB CANN) health.
#!/usr/bin/env bash
# Tinker AI — Qwen3.8-27B / Flash-Next / Muse Glimmer 30B & Orange Pi AI Station 96GB Runner (Guide #56)
set -euo pipefail
MODE="gpu_24gb_128k"
MODEL="./models/Qwen3.8-27B-Q4_K_M.gguf"
CTX="131072"
CPU_MOE="36"
PORT="8080"
if [ "$MODE" = "gpu_24gb_128k" ]; then
echo "=== [24GB-32GB GPU] Launching Qwen3.8-27B (16 KV layers) / Muse Glimmer 30B at $CTX Context ==="
exec llama-server \
-m "$MODEL" -ngl 99 -fa on \
--cache-type-k q8_0 --cache-type-v q8_0 \
-c "$CTX" --host 127.0.0.1 --port "$PORT"
elif [ "$MODE" = "flash_next_hybrid" ]; then
echo "=== [GPU + 128GB DDR5] Launching Qwen3.8-Flash-Next (6B Active + Host N-Gram/MoE Offload) ==="
exec llama-server \
-m "$MODEL" -ngl 99 --n-cpu-moe "$CPU_MOE" -fa on \
--cache-type-k fp8_e4m3 --cache-type-v q8_0 \
-c "$CTX" --host 127.0.0.1 --port "$PORT"
elif [ "$MODE" = "ascend_96gb" ]; then
echo "=== [Orange Pi AI Station 96GB] Auditing Ascend 310 CANN Runtime & Launching vllm-ascend ==="
npu-smi info
df -h / | awk 'NR==2 {print "Boot rootfs:", $1, $6}'
exec vllm serve "$MODEL" \
--device npu --dtype float16 \
--max-model-len 32768 --port "$PORT"
else
echo "Unknown RUN_MODE=$MODE (expected: gpu_24gb_128k | flash_next_hybrid | ascend_96gb)" >&2
exit 1
fi
Customizable Parameter Reference for run-qwen38-glimmer-or-ascend-station.sh
This SH artifact exposes 5 template variables compiled client-side in your browser.
| Template Token | Parameter Description | Verified Default | Example Format |
|---|---|---|---|
{{RUN_MODE}} | Target Mode (gpu_24gb_128k | flash_next_hybrid | ascend_96gb) | gpu_24gb_128k | gpu_24gb_128k |
{{MODEL_FILE}} | GGUF Path or Ascend Model Repo | ./models/Qwen3.8-27B-Q4_K_M.gguf | ./models/Qwen3.8-27B-Q4_K_M.gguf |
{{CTX_TOKENS}} | Context Window (-c) | 131072 | 131072 |
{{CPU_MOE_LAYERS}} | Host RAM MoE Layers (--n-cpu-moe for Flash-Next) | 36 | 36 |
{{LISTEN_PORT}} | HTTP API Port | 8080 | 8080 |
Verified Environment for "2026 Late-Fall 27B–30B Hybrid Models & High-RAM Edge Boards: Qwen3.8-27B, Qwen3.8-Flash-Next (51B N-Gram) & Muse Glimmer 30B on 24GB GPUs vs Orange Pi AI Station (96GB Ascend) & Radxa DragonStation"
RTX 4090 / 5080 SUPER (24GB) & RTX 5090 (32GB) + 128GB DDR5 vs Orange Pi AI Station (Ascend 310 · 96GB LPDDR4X · 176 TOPS) & Radxa DragonStation / DragonBay AI NAS (Qualcomm Hexagon NPU · 6x M.2 NVMe)
Qwen3.8-27B and Meta Muse Glimmer 30B ship under permissive Apache 2.0 licenses; Qwen3.8-Flash-Next uses the Qwen Community License 1.0.
- Orange Pi AI Station packs 48GB/96GB LPDDR4X-4266 (~68 GB/s theoretical bandwidth) and cannot boot its OS from the microSD/TF slot (requires onboard eMMC or M.2 PCIe 4.0 x4 NVMe); new hybrid operators like Gated DeltaNet or QSA lack native Ascend CANN fused kernels at launch.
- Qwen3.8-Flash-Next totals ~176B parameters (125B MoE + 51B N-gram embedding lookup), requiring ~102–115 GB of host DDR5 RAM or fast NVMe mmap even though per-token active compute is only 6B.
- Radxa DragonStation / DragonBay base Qualcomm Q8B-class Hexagon NPU (~29+ TOPS, up to 32GB RAM) targets low-wattage AI NAS indexing and 3B–8B SLMs unless paired with high-tier discrete PCIe AI accelerators.
3-Step Verification Checklist After Deploying run-qwen38-glimmer-or-ascend-station.sh
- Enabled
-fa onand--cache-type-k q8_0onllama.cpp v1.0.0+soQwen3.8-27B(16KV layers) orMuse Glimmer 30B(2KSWA +13global layers) stays within24GB–32GBVRAM at128K. - Confirmed
128GBhost DDR5 RAM before loadingQwen3.8-Flash-Next(125BMoE +51BN-gram) with--n-cpu-moeto avoid SSD swap thrashing. - Flashed Orange Pi AI Station OS onto M.2 PCIe 4.0 NVMe or onboard eMMC (never TF card) and verified CANN operator support before deploying new hybrid-attention models.