Configuration¶
There is no service to reconfigure and no port to forward. Access is the
ubuntu account on the instance.
Where things live¶
| Item | What it is |
|---|---|
deepeval |
Upstream CLI, on PATH. |
sushi-deepeval |
evaluate, read, and configure for the supplied example. |
/opt/deepeval/evaluate.py |
The two example criteria. |
/opt/deepeval/venv/bin/python |
Import deepeval from your own workspace. |
~/deepeval-results/ |
One report directory per run. |
~/.config/deepeval/judge.json |
Optional judge. Mode 0600. Keys base_url, model, api_key. |
/var/lib/deepeval |
Initialized state. Do not delete it to repair a run. |
Your own evaluations¶
Run deepeval --help for the upstream commands. Use the virtualenv Python
to create LLMTestCase values and select metrics. Keep the installed
runtime intact and use a separate environment when you add dependencies.
Features beyond the supplied example can need extra packages, a provider,
or a Confident AI account. Those are not included.
Judge file¶
sushi-deepeval configure will not replace an existing file. Edit
judge.json locally to change provider, model, or key. Do not paste the
key into a shell command, a log, a screenshot, or a support ticket.
chmod 600 ~/.config/deepeval/judge.json restores the mode if it was
widened.
The application under test and the judge are separate. The supplied answers are fixtures. A judge request sends the question, the actual answer, and the reference answer to your provider.
Last updated November 11, 2026 · Suggest an improvement