Interactive Deployment & Streaming API Troubleshooter
Static HTTP status tables miss runtime failures. Pick a failure scenario below, run the one-line terminal probes, and follow the decision tree to the verified fix.
Run the command below on your host machine. Is port 11434 bound to 127.0.0.1 or *:11434 (0.0.0.0)?
lsof -nP -iTCP:11434 -sTCP:LISTEN || ss -tlnp | grep 11434
If it shows 127.0.0.1:11434, requests from Docker bridge networks will be immediately refused.
6 Essential One-Line Terminal Probes for Local AI & Containers
Not sure which decision tree to pick first? Run these read-only terminal probes to isolate the exact failure layer in seconds.
| Layer | One-Line Diagnostic Command | Healthy Baseline | Failure Signal |
|---|---|---|---|
| 1. GPU Layer Offload | ollama ps | 100% GPU | Shows CPU/GPU split → VRAM OOM & Slow Generation |
| 2. Host Socket Binding | ss -tlnp | grep 11434 (or lsof -iTCP:11434) | *:11434 / 0.0.0.0:11434 | Only 127.0.0.1:11434 → Docker ECONNREFUSED |
| 3. Browser CORS Check | curl -i -X OPTIONS http://127.0.0.1:11434/api/tags -H "Origin: chrome-extension://test" | HTTP/1.1 204 / 200 | HTTP 403 Forbidden → 403 CORS & Reverse Proxy |
| 4. Embedding Dimension | curl -s http://127.0.0.1:11434/api/embeddings -d '{"model":"bge-m3","prompt":"a"}' | jq '.embedding|length' | 1024 (bge-m3) / 768 (nomic) | Mismatch vs DB collection → RAG Empty & Dimension |
| 5. Pi 5 Voltage & Thermal | vcgencmd get_throttled && lspci | throttled=0x0 | 0x50000 or empty lspci → Pi 5 Power & Hailo NPU |
| 6. Instant VRAM Flush | curl -s http://127.0.0.1:11434/api/generate -d '{"model":"qwen2.5:14b","keep_alive":0}' | done_reason: "unload" | VRAM still full → Check nvidia-smi for zombie processes |
4 Verified Root Causes & Fixes in "Container Cannot Reach Host Local Model (Ollama / vLLM)"
Fix: Bind Host Ollama to 0.0.0.0 (or Docker Bridge IP)
By default, Ollama binds strictly to 127.0.0.1:11434. Traffic arriving from a Docker bridge interface (172.17.0.x) is rejected by the OS TCP stack.
- On macOS: set OLLAMA_HOST via launchctl and restart the Ollama desktop app.
- On Linux systemd: add Environment="OLLAMA_HOST=0.0.0.0:11434" via systemctl edit ollama.service.
- Security check: ensure your host firewall blocks port 11434 on public LAN/WAN interfaces.
lsof -nP -iTCP:11434 -sTCP:LISTENFix: Replace localhost with host.docker.internal
Inside a container network namespace, localhost/127.0.0.1 refers to the container itself, not the physical host running your model.
- Update OLLAMA_BASE_URL or your HTTP node URL to http://host.docker.internal:11434.
- Ensure extra_hosts: ["host.docker.internal:host-gateway"] is present in docker-compose.yml.
docker exec tinker-open-webui wget -qO- http://host.docker.internal:11434/api/versionFix: Map host-gateway in Linux Docker Compose
Unlike Docker Desktop on Mac/Windows, native Linux Docker Engine does not inject host.docker.internal into /etc/hosts unless explicitly requested.
- Add extra_hosts: ["host.docker.internal:host-gateway"] to your service definition in docker-compose.yml.
- Recreate the container with docker compose up -d --force-recreate.
docker exec tinker-open-webui getent hosts host.docker.internalFix: Allow Docker Bridge Subnet Through Host Firewall (UFW / firewalld)
Linux UFW or iptables drops packets originating from the docker0 / br-* subnet (172.16.0.0/12) destined for host port 11434.
- Grant access strictly from the private Docker bridge CIDR (172.16.0.0/12) to port 11434.
- Reload UFW and test from inside the container again.
docker exec tinker-open-webui wget -qO- --timeout=5 http://host.docker.internal:11434/api/version