Skip to content

Getting started

Launch the document API, run the local retrieval demo, and open port 1416 only through a tunnel.

You need: an AWS account that can subscribe and launch EC2. Session Manager needs an instance profile with AmazonSSMManagedInstanceCore. SSH as ubuntu is an alternative. No model is required for retrieval.

1. Subscribe

Open the Haystack Document API listing, choose View purchase options, review the terms, and subscribe.

2. Launch the instance

  1. Choose t3.medium or larger, with at least 4 GiB of memory.
  2. Use an encrypted gp3 root volume of at least 20 GiB.
  3. Require IMDSv2.
  4. Leave inbound application ports closed.

3. Connect

haystack.service starts on boot. Configuration and the API token are created on the instance.

sudo cloud-init status --wait
sudo -iu ubuntu sushi-haystack status
sudo -iu ubuntu bash /opt/sushi/docs/demo.sh

The demo stores two synthetic documents, queries an Aurora retention window, requires the exact source that states 37 days, saves the response and document hashes under ~/haystack-results/, and rereads the documents through the API. It prints SUSHI_CUSTOMER_DEMO_PASSED only after those checks pass. The default run makes no model request.

A normal demo creates a new directory. The path ~/haystack-results/e2e/manifest.json applies when you ran sushi-haystack demo --output ~/haystack-results/e2e. Otherwise use the directory the demo printed:

sushi-haystack read ~/haystack-results/<run>/manifest.json

4. Open port 1416

The login banner prints this instance's forward command. From your workstation, forward guest port 1416 to a local port 1416:

aws ssm start-session --target <instance-id> \
  --document-name AWS-StartPortForwardingSession \
  --parameters '{"portNumber":["1416"],"localPortNumber":["1416"]}'

The API still requires its Bearer token. Read /etc/haystack/client.json only in an authenticated session. A workstation copy should be mode 0600. Do not print the token into a recording.

5. Generate an answer

Local retrieval works before you add a provider. To generate, edit /etc/haystack/config.json with sudoedit in an unrecorded session, as Configuration describes, restart the service, and run:

sudo -iu ubuntu bash /opt/sushi/docs/demo.sh --generate

That command requires the answer to contain 37 and to cite the fixture document id. It sends the question and the retrieved documents to your provider.

Last updated November 11, 2026 · Suggest an improvement