Troubleshoot vLLM¶
Separate first-boot runtime acquisition from model loading and inference. This reference describes the prepublication package, whose fresh-instance GPU and model validation is still pending.
The runtime download has not finished¶
Inspect the acquisition service:
sudo systemctl status sushi-vllm-runtime-acquire.service --no-pager
sudo journalctl -u sushi-vllm-runtime-acquire.service --no-pager
Allow for about 9.21 GB compressed plus unpacked disk space. Verify outbound DNS, HTTPS and vendor download availability. Anonymous Docker Hub shared-IP rate limits can produce HTTP 429; there is no fixed guaranteed allowance or fallback image. Acquisition is bounded to 30 minutes. Resolve connectivity, availability or storage problems before rebooting for the next automatic attempt. Do not manually substitute a runtime image.
The API is not ready¶
Select a model first, start vllm.service and run the supplied wait command.
First loading and compilation can take several minutes; the command allows up
to 15 minutes. Check GPU visibility and the service journal:
Use the documented GPU instance. CPU-only instances cannot run the server. For memory errors, choose a smaller compatible model within the fixed context, sequence and GPU-memory settings. Other architectures are not tuned automatically.
Model selection or startup validation fails¶
Use a physical local directory with readable config, tokenizer and safetensors files. Symlinks, Python/pickle files and remote model code are not supported. If files changed after selection, review them and their license and select the intended model again. Do not remove hash checks. The example helper refuses an existing final directory and preserves incomplete downloads separately; see the example recovery notes.
Requests return 401 or the client cannot connect¶
Use the instance key through a protected file or client credential store. The
supplied CLI reads it automatically. Confirm that the Session Manager tunnel
is open and that your API base is http://127.0.0.1:8000/v1. Port 8000 is an
API, not a browser chat page. Keep inbound access closed rather than exposing
it to work around a tunnel problem.
The saved walkthrough fails¶
The chosen model must follow the synthetic JSON and arithmetic instructions. A model that does not follow them fails the task. Keep the original responses and manifest; reread a completed result without new inference. These examples do not establish model accuracy, throughput or suitability for your workload.
Last updated November 16, 2026 · Suggest an improvement