Alternator Vector Search¶

CreateTable — VectorIndexes parameter¶

A new optional parameter VectorIndexes can be passed to CreateTable. It is a list of vector index definitions, each specifying:

VectorAttribute fields:

Projection fields:

The optional Projection parameter is identical to the one used for DynamoDB GSI and LSI, and specifies which attributes are stored in the vector index and returned when Select=ALL_PROJECTED_ATTRIBUTES is used in a vector search query:

Note: Currently only ProjectionType=KEYS_ONLY is implemented. Specifying ProjectionType=ALL or ProjectionType=INCLUDE returns a ValidationException. Since KEYS_ONLY is also the default, omitting Projection entirely is equivalent to specifying {'ProjectionType': 'KEYS_ONLY'}.

Example (using boto3):

table = dynamodb.create_table(
    TableName='my-table',
    KeySchema=[{'AttributeName': 'id', 'KeyType': 'HASH'}],
    AttributeDefinitions=[{'AttributeName': 'id', 'AttributeType': 'S'}],
    BillingMode='PAY_PER_REQUEST',
    VectorIndexes=[
        {
            'IndexName': 'embedding-index',
            'VectorAttribute': {'AttributeName': 'embedding', 'Dimensions': 1536},
        }
    ],
)

Constraints:

• A vector index may not share a name with another vector index, a GSI, or an LSI on the same table.

• The target attribute must not be a key column or an index key column.

• Dimensions must be a positive integer up to the implementation maximum.

• Vector indexes require ScyllaDB to operate with tablets (not vnodes).

• Multiple vector indexes can be created on the same table in a single CreateTable call.

UpdateTable — VectorIndexUpdates parameter¶

A new optional parameter VectorIndexUpdates can be passed to UpdateTable to add or remove a vector index after the table is created. At most one index operation (Create or Delete) may be requested per call.

Each element of the list is an object with exactly one of the following keys:

Create:

{
  "Create": {
    "IndexName": "my-vector-index",
    "VectorAttribute": {"AttributeName": "embedding", "Dimensions": 1536},
    "Projection": {"ProjectionType": "KEYS_ONLY"}
  }
}

The Projection field in the Create action is optional and accepts the same values as the Projection field in CreateTable’s VectorIndexes (see above). Currently only ProjectionType=KEYS_ONLY is supported; it is also the default when Projection is omitted.

Delete:

{
  "Delete": {
    "IndexName": "my-vector-index"
  }
}

The same constraints as CreateTable’s VectorIndexes apply.

DescribeTable — VectorIndexes in the response¶

DescribeTable (and CreateTable’s response) returns a VectorIndexes field in the TableDescription object when the table has at least one vector index. The structure mirrors the CreateTable input: a list of objects each containing IndexName, VectorAttribute (AttributeName + Dimensions), and Projection (ProjectionType). Currently Projection always contains {"ProjectionType": "KEYS_ONLY"} because that is the only supported projection type.

Each vector index entry also includes status fields that mirror the standard behavior of GlobalSecondaryIndexes in DynamoDB:

When creating a vector index on a non-empty table (via UpdateTable), data already in the table is picked up by the vector store through a full-table scan (backfill). During this period IndexStatus will be "CREATING" and Backfilling will be true. Once the scan completes the vector store transitions to monitoring CDC for ongoing changes, and IndexStatus becomes "ACTIVE".

Type enforcement before and after index creation differs:

• Pre-existing items: when the backfill scan encounters an item whose indexed attribute is not a list of exactly the right number of numbers (e.g., it is a string, a list of the wrong length, or contains non-numeric elements), that item is silently skipped and not indexed. The item remains in the base table unchanged.

• New writes once the index exists: any attempt to write a value to the indexed attribute that is not a list of exactly Dimensions numbers — where each number is representable as a 32-bit float — is rejected with a ValidationException. This applies to PutItem, UpdateItem, and BatchWriteItem. A missing value for the indexed attribute is always allowed; such items simply are not indexed.

Important: Applications must wait until IndexStatus is "ACTIVE" before issuing Query requests against a vector index. Queries on a vector index whose IndexStatus is still "CREATING" may fail. This applies both when adding a vector index to an existing table via UpdateTable and when creating a new table with a VectorIndexes parameter in CreateTable — even though the new table starts empty, the vector store still needs a short initialization period before it can serve queries.

Query — VectorSearch parameter¶

To perform a nearest-neighbor search, pass the VectorSearch parameter to Query. When this parameter is present the request is interpreted as a vector search rather than a standard key-condition query.

VectorSearch fields:

Example:

response = table.query(
    IndexName='embedding-index',
    Limit=10,
    VectorSearch={
        'QueryVector': {'L': [{'N': '0.1'}, {'N': '-0.3'}, {'N': '0.7'}, ...]},
    },
)

Requirements:

Differences from standard Query:

Vector search reinterprets several standard Query parameters in a fundamentally different way, and explicitly rejects others that have no meaningful interpretation:

• Limit means top-k, not page size. In a standard Query, Limit caps the number of items examined per page, and you page through results with ExclusiveStartKey. In vector search, Limit defines k: the ANN algorithm runs once and returns exactly the k nearest neighbors. There is no natural “next page” — each page would require a full re-run of the search — so ExclusiveStartKey is rejected.

• Results are ordered by vector distance, not by sort key. A standard Query returns rows in sort-key order; ScanIndexForward=false reverses that order. Vector search always returns results ordered by their distance to QueryVector (nearest first). Having ScanIndexForward specify sort-key direction has no meaning here, so ScanIndexForward is rejected.

• Eventual consistency only. The vector store is an external service fed asynchronously from ScyllaDB via CDC. Like GSIs, vector indexes can never reflect writes instantly, so strongly-consistent reads are impossible. ConsistentRead=true is rejected.

• No key condition. A standard Query requires a KeyConditionExpression to select which partition to read. Vector search queries the vector store globally across all partitions of the table. KeyConditions and KeyConditionExpression are therefore not applicable and are silently ignored. (Local vector indexes, which would scope the search to a single partition and use KeyConditionExpression, are not yet supported.)

Select parameter:

The standard DynamoDB Select parameter is supported for vector search queries and controls which attributes are returned for each matching item:

When neither Select nor ProjectionExpression/AttributesToGet is specified, Select defaults to ALL_PROJECTED_ATTRIBUTES. When ProjectionExpression or AttributesToGet is present without an explicit Select, it implies SPECIFIC_ATTRIBUTES. Using ProjectionExpression or AttributesToGet together with an explicit Select other than SPECIFIC_ATTRIBUTES is an error.

Note on performance: Unlike a DynamoDB LSI, a vector index allows you to read non-projected attributes (e.g., with ALL_ATTRIBUTES or SPECIFIC_ATTRIBUTES requesting a non-key column). However, doing so requires an additional read from the base table for each result — similar to reading through a secondary index (rather than a materialized view) in CQL — and is therefore significantly slower than returning only projected attributes with ALL_PROJECTED_ATTRIBUTES. For latency-sensitive applications, prefer ALL_PROJECTED_ATTRIBUTES or limiting SPECIFIC_ATTRIBUTES to key columns.

FilterExpression:

Vector search supports FilterExpression for post-filtering results. This works the same way as FilterExpression on a standard DynamoDB Query: after the ANN search, the filter is applied to each candidate item and only matching items are returned.

Important: filtering happens after the Limit nearest neighbors have already been selected by the vector index. If the filter discards some of those candidates, the response may contain fewer than Limit items. The server does not automatically fetch additional neighbors to replace filtered-out items. This is identical to how FilterExpression interacts with Limit in a standard DynamoDB Query.

The response always includes two count fields:

Interaction with Select:

• Select=ALL_ATTRIBUTES: Each candidate item is fetched from the base table, the filter is evaluated against all its attributes, and only matching items are returned. Count reflects the number of items that passed the filter.

• Select=SPECIFIC_ATTRIBUTES: Each candidate item is fetched from the base table — including any attributes needed by the filter expression, even if those attributes are not listed in ProjectionExpression — and the filter is applied. Only the projected attributes are returned in the response; filter attributes that were not requested are not included in the returned items.

• Select=COUNT: The candidate items are still fetched from the base table and the filter is evaluated for each one, but no Items list is returned. Count reflects the number of items that passed the filter; ScannedCount is the total number of candidates examined. This is useful for counting matches without transferring item data to the client.

• Select=ALL_PROJECTED_ATTRIBUTES (default): When no filter is present this is the most efficient mode — results are returned directly from the vector store without any base-table reads. When a FilterExpression is present, however, the full item must be fetched from the base table to evaluate the filter, and only the projected (key) attributes are returned for items that pass.

Note: QueryFilter (the legacy non-expression filter API) is not supported for vector search queries and will be rejected with a ValidationException. Use FilterExpression instead.