TinkerLab
ARTIFACT // RUN-QWEN38-GLIMMER-OR-ASCEND-STATION.SH (SH)

Qwen3.8-27B / Flash-Next (51B N-Gram Host Cache) / Muse Glimmer 30B & Ascend 96GB Runner (run-qwen38-glimmer-or-ascend-station.sh)

Runs Qwen3.8-27B (16 KV layers) or Muse Glimmer 30B (2K SWA + global attn) at 128K context on 24GB–32GB GPUs, offloads Qwen3.8-Flash-Next N-gram/MoE tables to DDR5 RAM, or audits Orange Pi AI Station (Ascend 310 96GB CANN) health.

[KEY VERIFIED TAKEAWAY / TL;DR]This reproducible configuration artifact "Qwen3.8-27B / Flash-Next (51B N-Gram Host Cache) / Muse Glimmer 30B & Ascend 96GB Runner" (run-qwen38-glimmer-or-ascend-station.sh, SH) is paired with Tinker AI Lab's hardware-verified guide "qwen38-27b-flash-next-muse-glimmer-30b-vs-orangepi-ai-station-96gb-radxa-dragonstation". It exposes 5 customizable browser variables (RUN_MODE=gpu_24gb_128k, MODEL_FILE=./models/Qwen3.8-27B-Q4_K_M.gguf, CTX_TOKENS=131072, CPU_MOE_LAYERS=36, LISTEN_PORT=8080) with loopback-first security defaults.
01 // HOMELABLocal AI / HomeLab· Verified: 2026-11-18
Browse All 60 Artifacts →
run-qwen38-glimmer-or-ascend-station.sh

Qwen3.8-27B / Flash-Next (51B N-Gram Host Cache) / Muse Glimmer 30B & Ascend 96GB Runner

Runs Qwen3.8-27B (16 KV layers) or Muse Glimmer 30B (2K SWA + global attn) at 128K context on 24GB–32GB GPUs, offloads Qwen3.8-Flash-Next N-gram/MoE tables to DDR5 RAM, or audits Orange Pi AI Station (Ascend 310 96GB CANN) health.

View Paired Guide →
Customizable Variables (Saved in Browser)
UTF-8 Encoded · Live Client-Side Compilation
#!/usr/bin/env bash
# Tinker AI — Qwen3.8-27B / Flash-Next / Muse Glimmer 30B & Orange Pi AI Station 96GB Runner (Guide #56)
set -euo pipefail

MODE="gpu_24gb_128k"
MODEL="./models/Qwen3.8-27B-Q4_K_M.gguf"
CTX="131072"
CPU_MOE="36"
PORT="8080"

if [ "$MODE" = "gpu_24gb_128k" ]; then
  echo "=== [24GB-32GB GPU] Launching Qwen3.8-27B (16 KV layers) / Muse Glimmer 30B at $CTX Context ==="
  exec llama-server \
    -m "$MODEL" -ngl 99 -fa on \
    --cache-type-k q8_0 --cache-type-v q8_0 \
    -c "$CTX" --host 127.0.0.1 --port "$PORT"
elif [ "$MODE" = "flash_next_hybrid" ]; then
  echo "=== [GPU + 128GB DDR5] Launching Qwen3.8-Flash-Next (6B Active + Host N-Gram/MoE Offload) ==="
  exec llama-server \
    -m "$MODEL" -ngl 99 --n-cpu-moe "$CPU_MOE" -fa on \
    --cache-type-k fp8_e4m3 --cache-type-v q8_0 \
    -c "$CTX" --host 127.0.0.1 --port "$PORT"
elif [ "$MODE" = "ascend_96gb" ]; then
  echo "=== [Orange Pi AI Station 96GB] Auditing Ascend 310 CANN Runtime & Launching vllm-ascend ==="
  npu-smi info
  df -h / | awk 'NR==2 {print "Boot rootfs:", $1, $6}'
  exec vllm serve "$MODEL" \
    --device npu --dtype float16 \
    --max-model-len 32768 --port "$PORT"
else
  echo "Unknown RUN_MODE=$MODE (expected: gpu_24gb_128k | flash_next_hybrid | ascend_96gb)" >&2
  exit 1
fi
PARAMETER SPECIFICATION // 参数化变量规范说明表

Customizable Parameter Reference for run-qwen38-glimmer-or-ascend-station.sh

This SH artifact exposes 5 template variables compiled client-side in your browser.

Template TokenParameter DescriptionVerified DefaultExample Format
{{RUN_MODE}}Target Mode (gpu_24gb_128k | flash_next_hybrid | ascend_96gb)gpu_24gb_128kgpu_24gb_128k
{{MODEL_FILE}}GGUF Path or Ascend Model Repo./models/Qwen3.8-27B-Q4_K_M.gguf./models/Qwen3.8-27B-Q4_K_M.gguf
{{CTX_TOKENS}}Context Window (-c)131072131072
{{CPU_MOE_LAYERS}}Host RAM MoE Layers (--n-cpu-moe for Flash-Next)3636
{{LISTEN_PORT}}HTTP API Port80808080
PAIRED HARDWARE MANIFEST // 配套实测环境与边界记录

Verified Environment for "2026 Late-Fall 27B–30B Hybrid Models & High-RAM Edge Boards: Qwen3.8-27B, Qwen3.8-Flash-Next (51B N-Gram) & Muse Glimmer 30B on 24GB GPUs vs Orange Pi AI Station (96GB Ascend) & Radxa DragonStation"

Tested Hardware & OS

RTX 4090 / 5080 SUPER (24GB) & RTX 5090 (32GB) + 128GB DDR5 vs Orange Pi AI Station (Ascend 310 · 96GB LPDDR4X · 176 TOPS) & Radxa DragonStation / DragonBay AI NAS (Qualcomm Hexagon NPU · 6x M.2 NVMe)

Pinned Stack & Model Versions
llama.cpp v1.0.0+ (GDN / QSA / SWA)vLLM 0.12+ / vllm-ascend (CANN 8.0+)openEuler 22.03 / Ubuntu 24.04Fygo OS (Radxa AI NAS)
Offline Boundary (100% Local Single-GPU, Hybrid DDR5 & Edge AI NAS)

Qwen3.8-27B and Meta Muse Glimmer 30B ship under permissive Apache 2.0 licenses; Qwen3.8-Flash-Next uses the Qwen Community License 1.0.

Known Operational Limits & Failure Modes
  • Orange Pi AI Station packs 48GB/96GB LPDDR4X-4266 (~68 GB/s theoretical bandwidth) and cannot boot its OS from the microSD/TF slot (requires onboard eMMC or M.2 PCIe 4.0 x4 NVMe); new hybrid operators like Gated DeltaNet or QSA lack native Ascend CANN fused kernels at launch.
  • Qwen3.8-Flash-Next totals ~176B parameters (125B MoE + 51B N-gram embedding lookup), requiring ~102–115 GB of host DDR5 RAM or fast NVMe mmap even though per-token active compute is only 6B.
  • Radxa DragonStation / DragonBay base Qualcomm Q8B-class Hexagon NPU (~29+ TOPS, up to 32GB RAM) targets low-wattage AI NAS indexing and 3B–8B SLMs unless paired with high-tier discrete PCIe AI accelerators.
DEPLOYMENT CHECKLIST // 配置落地核验步骤

3-Step Verification Checklist After Deploying run-qwen38-glimmer-or-ascend-station.sh

  1. Enabled -fa on and --cache-type-k q8_0 on llama.cpp v1.0.0+ so Qwen3.8-27B (16 KV layers) or Muse Glimmer 30B (2K SWA + 13 global layers) stays within 24GB–32GB VRAM at 128K.
  2. Confirmed 128GB host DDR5 RAM before loading Qwen3.8-Flash-Next (125B MoE + 51B N-gram) with --n-cpu-moe to avoid SSD swap thrashing.
  3. Flashed Orange Pi AI Station OS onto M.2 PCIe 4.0 NVMe or onboard eMMC (never TF card) and verified CANN operator support before deploying new hybrid-attention models.