Operate vLLM¶
Manage model serving, protect stored keys and preserve the original saved responses. This reference describes the prepublication package; fresh-instance GPU and model validation is pending.
Services and GPU diagnostics¶
Check the server and recent logs:
sudo systemctl status vllm.service --no-pager
sudo journalctl -u vllm.service --since '15 minutes ago' --no-pager
Check GPU and driver visibility:
Stop serving before changing a model, credentials or taking a consistent backup:
After configuration or backup, start and wait:
The automatic first-boot acquisition is a separate service,
sushi-vllm-runtime-acquire.service. Inspect its journal for runtime download
errors before starting the model server. Do not substitute a mutable image or
manually pull a different runtime to bypass acquisition checks.
Reread saved responses¶
Switch to ubuntu:
Reread an existing directory without new inference:
read -r -p 'Full saved result directory: ' vllm_saved_run
/usr/local/bin/sushi-vllm read --output "$vllm_saved_run"
Keep the manifest, JSON generation response, streaming response and concurrent responses together. The checker verifies hashes and recomputes expected totals. A failed example remains useful diagnostic output; do not edit it into a pass.
Persistence and backups¶
Back up /etc/vllm/, /var/lib/vllm/models/, /var/lib/vllm/cache/ and
/home/ubuntu/vllm-results/ to encrypted storage with restricted readers. The
cached runtime resides in /var/lib/docker; retaining the complete root EBS
volume also preserves it. Allow enough storage for unpacked runtime layers,
models and caches. Termination can delete the root volume.
Normal serving uses the cached runtime and selected local model without fetching new weights. Keep endpoint access private and review logs and saved responses for sensitive content before sharing. Your own prompts and models can contain confidential data even though the supplied example uses synthetic requests.
Updates and support¶
Use a reviewed replacement image for runtime or GPU dependency updates.
Arbitrary pip upgrades or image pulls can invalidate the managed configuration.
Model changes require license review and reselection through the supplied helper.
Contact Software Sushi with the package version, symptom and redacted service output. Never send the API key, private model files or prompts. Technical-support coverage requires a separate agreement; commercial upstream support is not bundled.
Last updated November 16, 2026 · Suggest an improvement