Skip to content
LiteLLM Proxy AI Platform & Data · LiteLLM 1.104.2 Launch on AWS

Operations

This page covers running the gateway day to day: how it is secured, how to manage the service, where logs go, what to back up, how updates work and how to get help.

Security model

  • Loopback and HTTPS only. The proxy listens on 127.0.0.1:4000 with TLS. Nothing listens on a public interface.
  • Master key. Chat and other API calls need the master key as a bearer token. Only the health endpoints answer without it.
  • Per-instance secrets. The certificate, private key and master key are created on first boot. The image contains none of them, and no provider credentials.
  • File permissions. The master key is readable by root only. The ubuntu user cannot read the master key or the TLS private key. The litellm service user can read the certificate, key, config and runtime environment.
  • Service sandbox. The litellm-proxy service runs as the litellm user and can write only to /var/lib/litellm-proxy.

Keep the security group closed to inbound traffic, and connect with Session Manager or SSH restricted to your administrative range. Watch your provider spending: every call through the gateway is billed by the provider.

Manage the service

sudo systemctl status litellm-proxy
sudo systemctl restart litellm-proxy
sudo systemctl stop litellm-proxy

The first-boot service, sushi-litellm-proxy-firstboot, runs before the proxy on every boot. It creates anything that is missing, restores file permissions and rewrites runtime.env.

Check health:

sudo curl -sS --cacert /var/lib/litellm-proxy/tls/cert.pem \
  https://127.0.0.1:4000/health/liveliness

Logs

The proxy logs to the systemd journal:

sudo journalctl -u litellm-proxy -n 100 --no-pager
sudo journalctl -u sushi-litellm-proxy-firstboot --no-pager

Backups

Back up these directories together, over a private channel:

  • /etc/litellm-proxy/ (configuration and your provider key file)
  • /var/lib/litellm-proxy/ (master key and TLS files)

If you added a systemd drop-in, also keep /etc/systemd/system/litellm-proxy.service.d/. Restore them with the same ownership and permissions before the proxy starts. Never put keys in a ticket.

Updates

  • Ubuntu packages, including OpenSSL, receive unattended security updates.
  • LiteLLM itself is not updated by apt or pip. A new LiteLLM release comes as a new version of the image. There is no in-place upgrade: launch the new version, copy the files above, and point your clients or reverse proxy at the new instance.

Support

Software Sushi supports the packaging, first-boot setup and this documentation. Email info@softwaresushi.com with your product version, the steps you took and the error. Never include the master key, provider keys or private configuration. Guaranteed response times require a separate agreement. BerriAI support and enterprise features are not included.

Last updated October 13, 2026 · Suggest an improvement