Skip to content

Usage

Send one bounded query and passage pool. Resolve returned IDs against that exact pool, then inspect the original source before using ranked context.

Request

{
  "query": "How do I update my billing address?",
  "documents": [
    {"id": "billing_steps", "text": "Open Settings, choose Billing, edit the address, and save your changes."},
    {"id": "delivery", "text": "Delivery tracking appears in your order history."}
  ],
  "top_k": 2
}

Use application/json, or application/jsonlines with one object per invocation. A request has exactly query, documents and top_k. Multiple JSON objects in one inference invocation are rejected.

Bounds

Input Limit
Documents 1 to 16; each has exactly id and text
IDs Unique opaque values matching [A-Za-z0-9_-]{1,64}
Text Nonblank; at most 16,000 Unicode code points per query or passage
Combined text 32,000 query-plus-passage code points
HTTP body 96 KiB
Complete query/passage pair 512 actual tokens, including special tokens
top_k Integer from 1 through the supplied document count

Both the query and passage consume the pair budget. There is no silent truncation. Chunk long documents upstream and evaluate how chunking affects your task.

Response

results contains the original id, zero-based input index and raw relevance score, ordered by descending score. Equal scores break ties by ID. pair_tokens follows input order. The response also identifies the model, version and original upstream revision.

review_required is always true, truncated is false, and answerability_measured is false. Scores are uncalibrated logits, not probabilities or verified-answer decisions. Do not invent a shared confidence threshold to detect missing answers.

Errors and retries

Status Action
400 Correct malformed JSON or schema-invalid input.
413 Reduce the body size.
415 Use a supported media type without unsupported content encoding.
422 Shorten or split complete query/passage pairs upstream.
429 Retry busy capacity with bounded exponential backoff.
500 Inspect the redacted error and request ID; contact support if it persists.

SageMaker may wrap container errors in a 424 ModelError. Inspect OriginalStatusCode. Avoid logging sensitive request bodies in your client. See limits and evaluation before integrating the output into a decision or answer-generation workflow.

Last updated November 15, 2026 · Suggest an improvement