Getting started¶
This page takes you from the AWS Marketplace listing to a signed-in AnythingLLM workspace that answers questions about your documents.
You need: an AWS account with permission to subscribe to AWS Marketplace products, launch EC2 instances and use AWS Systems Manager, and, on your workstation, the AWS CLI with the Session Manager plugin. To chat, you also need access to an OpenAI-compatible model endpoint, such as Amazon Bedrock.
1. Subscribe¶
Open the AnythingLLM RAG Workbench listing in AWS Marketplace, choose View purchase options, review the terms and subscribe. When the subscription is confirmed, choose Launch your software.
2. Launch the instance¶
On the EC2 launch page:
- Keep the instance type at t3.medium, or choose one with at least 4 GiB of memory.
- Under Configure storage, use an encrypted gp3 root volume of at least 40 GiB.
- Under Network settings, allow no inbound traffic. AnythingLLM listens only on the instance itself, and Session Manager does not need an open port. If you also want SSH, allow port 22 from your administrative address range only. Keep outbound HTTPS open so AnythingLLM can reach your model providers.
- Under Advanced details, choose an IAM instance profile that includes
the AWS managed policy
AmazonSSMManagedInstanceCore, and require IMDSv2. - Choose Launch instance.
Do not open port 3001 to the internet
It listens on loopback only. You reach the UI through Session Manager port forwarding, so there is nothing to open.
3. Connect and wait for first boot¶
In the EC2 console, select the instance, choose Connect, then Session Manager, then Connect. Or, from your workstation:
Switch to the ubuntu user and wait for first boot to finish:
sudo -iu ubuntu
cloud-init status --wait
systemctl is-active sushi-anythingllm-firstboot anythingllm
Both services report active once the container is started. It can take a
little longer before the UI answers.
4. Run the walkthrough¶
It checks that the service is running and bound to loopback, that a wrong
password is refused and the right one accepted, that documents survive a
restart, and that the web page collector works. It restarts AnythingLLM once.
It ends with SUSHI_CUSTOMER_DEMO_PASSED.
5. Get the admin password¶
The password is stored only on the instance, readable by root:
Keep it out of tickets, chat and screen recordings.
6. Open the UI¶
On your workstation, start a port-forwarding session. Replace
<instance-id> and use the instance's region:
aws ssm start-session \
--target <instance-id> \
--document-name AWS-StartPortForwardingSession \
--parameters '{"portNumber":["3001"],"localPortNumber":["3001"]}'
Leave it running, open http://127.0.0.1:3001 in your browser and sign in
with the password from step 5.
7. Connect a model¶
In AnythingLLM, open Settings, then LLM, choose Generic OpenAI and enter any OpenAI-compatible endpoint, its API key and a model name.
For Amazon Bedrock, use its OpenAI-compatible endpoint with a Bedrock API key:
| Field | Value |
|---|---|
| Base URL | https://bedrock-runtime.<region>.amazonaws.com/openai/v1 |
| API key | a Bedrock API key |
| Model | a Bedrock model that supports this endpoint, such as openai.gpt-oss-20b-1:0 |
The built-in AWS Bedrock provider needs keys
AnythingLLM's own AWS Bedrock option asks for access keys and does not use the instance's IAM role. The Generic OpenAI route above avoids storing long-lived AWS access keys.
Also choose an embedding provider under Settings, so AnythingLLM can index your documents.
8. Chat with a document¶
- Create a workspace.
- New workspaces start in automatic (agent) mode. For plain question and answer over your documents, open the workspace settings and set the chat mode to Chat.
- Upload a document, or add a web page, and move it into the workspace.
- Ask a question about it.
Next: Configuration.
Last updated October 13, 2026 · Suggest an improvement