Skip to content
BentoML Prediction Server AI Platform & Data · BentoML 1.4.39 Launch on AWS

Getting started

Launch the BentoML server, run the sample, and open Swagger.

You need: an AWS account that can subscribe, launch EC2, and start Session Manager. Install the AWS CLI and the Session Manager plugin on your workstation.

Watch on YouTube.

1. Subscribe

Open the BentoML listing, choose View purchase options, review the terms, and subscribe. When the subscription is active, choose Launch your software.

2. Launch the instance

  1. Choose t3.large with at least 4 GiB of memory.
  2. Use an encrypted gp3 root volume of at least 32 GiB.
  3. Require IMDSv2.
  4. Attach an instance profile that can use Session Manager.
  5. Leave inbound application ports closed. If you also use SSH, limit TCP 22 to an administrative address range you choose.
  6. Choose Launch instance.

3. Run the walkthrough

Connect with Session Manager as ubuntu and wait for cloud-init:

sudo -iu ubuntu bash /opt/sushi/docs/demo.sh

4. Open Swagger

As ubuntu, start the example UI:

sushi-bentoml-example

From your computer, forward port 3000 and open it in the browser. In Swagger UI, pick an endpoint, choose Try it out, then Execute.

aws ssm start-session --target <instance-id> \
  --document-name AWS-StartPortForwardingSession \
  --parameters '{"portNumber":["3000"],"localPortNumber":["3000"]}'

5. Call the prediction API

The built-in prediction API uses port 8448 and a per-instance token. sushi-bentoml-api reads the access file for you. Do not print the token. This image does not call a model provider.

aws ssm start-session --target <instance-id> \
  --document-name AWS-StartPortForwardingSession \
  --parameters '{"portNumber":["8448"],"localPortNumber":["8448"]}'

The helper's arguments are in /opt/sushi/docs/README.md. See Configuration.

Last updated November 1, 2026 · Suggest an improvement