Skip to content

Get started with DSPy

Configure an endpoint and run the included classifier workflow from a private terminal. You control the model endpoint, the data you send and its budget.

Prepublication guide

Package 0.1.0 remains Limited and is not available for public subscription. These instructions describe the image for authorized users with access; they are not a public launch offer. See availability and scope.

1. Connect and check setup

The documented configuration is t3.medium, x86_64, 4 GiB RAM and at least 24 GiB encrypted root storage. Use IMDSv2 and an instance role/network that supports Session Manager. No application inbound port is required. Allow outbound HTTPS to your model endpoint. Open a Session Manager shell:

sudo cloud-init status --wait
sudo systemctl status sushi-dspy-firstboot.service --no-pager

First boot creates /home/ubuntu/dspy-workspace/ and /home/ubuntu/dspy-results/, owned by ubuntu with mode 0700.

2. Configure the endpoint

Switch from the Session Manager account to ubuntu:

sudo -iu ubuntu

In a private interactive terminal, run:

/opt/dspy/venv/bin/python -I -B /opt/dspy/configure.py

Enter a model identifier beginning with openai/, your endpoint's full OpenAI-compatible HTTPS API base and the hidden API key. Use the API base your endpoint documents; unlike NextChat's root-URL field, this is the API base. The helper creates ~/dspy-workspace/provider.json with mode 0600 and refuses to replace an existing file. No endpoint, model or account is created for you.

Requests leave the instance

Review the endpoint's data policy, access and cost before running the task. Use synthetic or appropriately authorized tickets. The workflow needs your available endpoint; inference and endpoint infrastructure are not bundled.

3. Run the walkthrough

sudo -iu ubuntu bash /opt/sushi/docs/demo.sh

The included dataset makes 30 sequential SDK requests. Allow roughly 1–5 minutes, depending on the endpoint. Each request uses zero temperature, a 128-token output limit and a 30-second timeout, with no configured SDK retries or adapter fallback requests. A failed task is not automatically retried at your expense.

The CLI compares a zero-shot program and LabeledFewShot candidates with up to 3 or 6 training demonstrations, capped by available training rows. Validation accuracy selects the program; ties favor the earlier, smaller candidate. Heldout examples are excluded from optimization and selection. The baseline and reloaded selected program are then evaluated on heldout examples.

4. Inspect the saved result

A new folder under /home/ubuntu/dspy-results/ contains dataset.json, validation.json, program.json, heldout-before.json, heldout-after.json and result.json. The independent reader checks their hashes, recomputes metrics and verifies that saved demonstrations came only from training.

SUSHI_CUSTOMER_DEMO_PASSED means the workflow and readback completed. Read the actual scores; it does not promise better accuracy or business suitability. Continue with custom datasets and Python programs.

Last updated November 16, 2026 · Suggest an improvement