# Collections and data model
Source: https://docs.lambdadb.ai/guides/collections/collection-overview
Understand the LambdaDB data model where projects contain collections of flexible documents, similar to tables and rows in a relational database.
A LambdaDB collection is a logical namespace that holds a set of documents, where each document is a set of fields — which, in turn, are key-value pairs that contain your data.
## Database structure comparison
LambdaDB's structure differs from traditional relational databases. Here's how they compare:
| RDBMS | LambdaDB |
| :--------- | :---------------------------- |
| Database | Project |
| Table | Collection |
| Column/row | Document with key-value pairs |
Think of a LambdaDB project as a database that can contain many collections (similar to tables), and within each collection, you have many documents containing your actual data as flexible key-value pairs.
Indexes in LambdaDB are not the same as you'd find in a relational database. The whole document is stored as is regardless of its existence in index configurations, but being stored does not necessarily mean it is searchable.
## API interaction
LambdaDB provides a RESTful JSON-based API for interacting with document data. You can perform the following operations by sending HTTP requests to the appropriate endpoints:
* **Upsert** documents into collections
* **Search** across documents using various query types
* **Delete** individual documents or entire collections
* **Update** document data and collection configurations
These CRUD-like operations can take place at both individual document level and collection level, giving you flexibility in how you manage your data.
## Zero-copy branching
Zero-copy collection fork is a powerful feature that allows you to create a new collection by referencing an existing collection's data, without physically copying the underlying data.
This feature is useful when you want to create a new collection with the same configurations and data as an existing collection,
in order to load-test, develop, and experiment in an isolated environment.
### Key benefits
* **Instant creation**: Collections are created immediately without waiting for data transfer.
* **Storage efficiency**: No additional storage space required for the forked collection.
* **Isolated environment**: Perfect for load testing, development, and experimentation.
* **Same configurations**: Inherits all index configurations and data from the source collection.
### Use cases
This feature is particularly useful when you want to:
* Create isolated environments for testing without affecting production data.
* Set up staging environments that mirror production collections.
* Experiment with different query patterns or configurations.
* Perform load testing with real data volumes.
* Create backup references for disaster recovery scenarios.
Changes made to documents in a forked collection do not affect the original collection, ensuring complete isolation while maintaining efficiency.
# Create a new collection
Source: https://docs.lambdadb.ai/guides/collections/create-a-collection
Create LambdaDB collections with index configurations, partitioning, and point-in-time restore using Python, TypeScript, Go, or the REST API.
This page shows you how to create a collection with various configurations.
## Parameters
| Parameter | Type | Required | Description |
| :------------------- | :----- | :------- | :------------------------------------------------------------------------------------------------ |
| collectionName | string | ✓ | Collection name must be unique within a project and the supported maximum length is 52 characters |
| indexConfigs | object | | Field configuration for indexing. Required when creating from scratch |
| sourceProjectName | string | | Source project name for PITR or branching operations |
| sourceCollectionName | string | | Source collection name for PITR or branching operations |
| sourceDatetime | string | | ISO 8601 formatted datetime for PITR (e.g., "2024-01-15T10:30:00Z") |
| sourceProjectApiKey | string | | API key for the source project |
| partitionConfig | object | | Partition configuration |
## Create a collection from scratch
The simplest way to create a collection is to provide the collection name and the field [index configurations](/guides/collections/index-types).
```python Python theme={null}
from lambdadb import LambdaDB, models
with LambdaDB(
project_api_key="YOUR_API_KEY",
base_url="YOUR_BASE_URL",
project_name="YOUR_PROJECT_NAME",
) as client:
basic_config = {
"url": {"type": "keyword"},
"author": {"type": "keyword"},
"content": {
"type": "text",
"analyzers": ["japanese", "korean", "english"],
},
}
client.collections.create(collection_name="example-collection", index_configs=basic_config)
```
Python: `LambdaDB` supports context manager usage. `__enter__` returns the client, and `__exit__` calls `client.close()` (closing the SDK-owned HTTP client) and makes the client unusable after the `with` block. If you don't use `with`, call `client.close()` when you're done. If you pass a custom `client=`/`async_client=`, you own closing it.
```typescript TypeScript theme={null}
import { LambdaDBClient } from "@functional-systems/lambdadb";
const client = new LambdaDBClient({
projectApiKey: "YOUR_API_KEY",
baseUrl: "YOUR_BASE_URL",
projectName: "YOUR_PROJECT_NAME",
});
await client.createCollection({
collectionName: "example-collection",
indexConfigs: {
url: { type: "keyword" },
author: { type: "keyword" },
content: { type: "text", analyzers: ["japanese", "korean", "english"] },
},
});
```
```go Go theme={null}
client := lambdadb.New(
lambdadb.WithBaseURL("YOUR_BASE_URL"),
lambdadb.WithProjectName("YOUR_PROJECT_NAME"),
lambdadb.WithAPIKey("YOUR_API_KEY"),
)
_, err := client.Collections.Create(ctx, lambdadb.CreateCollectionOptions{
CollectionName: "example-collection",
IndexConfigs: map[string]interface{}{
"url": map[string]interface{}{"type": "keyword"},
"author": map[string]interface{}{"type": "keyword"},
"content": map[string]interface{}{"type": "text", "analyzers": []string{"japanese", "korean", "english"}},
},
})
```
```bash cURL theme={null}
LAMBDADB_PROJECT_API_KEY="YOUR_API_KEY"
curl -i -X POST "$BASE_URL/projects/$PROJECT_NAME/collections" \
-H "content-type: application/json" \
-H "x-api-key: $LAMBDADB_PROJECT_API_KEY" \
-d '{
"collectionName": "example-collection",
"indexConfigs": {
"url": { "type": "keyword" },
"author": { "type": "keyword" },
"content": { "type": "text", "analyzers": ["japanese", "korean", "english"] }
}
}'
```
## Create a collection with managed embeddings
Managed embeddings let LambdaDB derive vector values from a source text field. Define the vector field with `managedEmbedding: true` and an `embedding` block. The source field must be a `text` field in the same collection.
For the current provider and model matrix, see [Managed embeddings](/guides/collections/managed-embeddings).
```python Python theme={null}
from lambdadb import LambdaDB
with LambdaDB(
project_api_key="YOUR_API_KEY",
base_url="YOUR_BASE_URL",
project_name="YOUR_PROJECT_NAME",
) as client:
client.collections.create(
collection_name="semantic-docs",
index_configs={
"body": {
"type": "text",
"analyzers": ["english"],
},
"bodyEmbedding": {
"type": "vector",
"managedEmbedding": True,
"embedding": {
"provider": "openai",
"model": "text-embedding-3-small",
"sourceField": "body",
},
},
},
)
```
```typescript TypeScript theme={null}
await client.createCollection({
collectionName: "semantic-docs",
indexConfigs: {
body: {
type: "text",
analyzers: ["english"],
},
bodyEmbedding: {
type: "vector",
managedEmbedding: true,
embedding: {
provider: "openai",
model: "text-embedding-3-small",
sourceField: "body",
},
},
},
});
```
```bash cURL theme={null}
curl -i -X POST "$BASE_URL/projects/$PROJECT_NAME/collections" \
-H "content-type: application/json" \
-H "x-api-key: $LAMBDADB_PROJECT_API_KEY" \
-d '{
"collectionName": "semantic-docs",
"indexConfigs": {
"body": {
"type": "text",
"analyzers": ["english"]
},
"bodyEmbedding": {
"type": "vector",
"managedEmbedding": true,
"embedding": {
"provider": "openai",
"model": "text-embedding-3-small",
"sourceField": "body"
}
}
}
}'
```
For managed embedding vector fields, do not send top-level `dimensions` or `similarity` in the field config. LambdaDB resolves and stores those values under `embedding`.
## Create a partitioned collection
LambdaDB supports hash-based partitioning for a specified field.
```python Python theme={null}
from lambdadb import LambdaDB, models
with LambdaDB(project_api_key="YOUR_API_KEY", base_url="YOUR_BASE_URL", project_name="YOUR_PROJECT_NAME") as client:
basic_config = {
"url": {"type": "keyword"},
"author": {"type": "keyword"},
"content": {"type": "text", "analyzers": ["japanese", "korean", "english"]},
}
client.collections.create(
collection_name="example-collection",
index_configs=basic_config,
partition_config={"field_name": "url", "data_type": "keyword", "num_partitions": 4},
)
```
```typescript TypeScript theme={null}
await client.createCollection({
collectionName: "example-collection",
indexConfigs: { url: { type: "keyword" }, author: { type: "keyword" }, content: { type: "text", analyzers: ["japanese", "korean", "english"] } },
partitionConfig: { fieldName: "url", dataType: "keyword", numPartitions: 4 },
});
```
```bash cURL theme={null}
curl -i -X POST "$BASE_URL/projects/$PROJECT_NAME/collections" \
-H "content-type: application/json" \
-H "x-api-key: $LAMBDADB_PROJECT_API_KEY" \
-d '{
"collectionName": "example-collection",
"indexConfigs": { "url": {"type": "keyword"}, "author": {"type": "keyword"}, "content": {"type": "text", "analyzers": ["japanese", "korean", "english"]} },
"partitionConfig": { "fieldName": "url", "dataType": "keyword", "numPartitions": 4 }
}'
```
Currently, only `keyword` type is supported for partitioning.
## Point-in-time recovery (PITR)
LambdaDB automatically maintains continuous backups at the collection level with a default retention period of 30 days.
You can create a collection from a specific point in time using PITR functionality.
```python Python theme={null}
from lambdadb import LambdaDB
with LambdaDB(project_api_key="YOUR_TARGET_API_KEY", base_url="YOUR_BASE_URL", project_name="YOUR_PROJECT_NAME") as client:
restored_collection = client.collections.create(
collection_name="restored-collection",
source_project_name="source-project-name",
source_project_api_key="YOUR_SOURCE_API_KEY",
source_collection_name="source-collection-name",
source_datetime="2024-01-15T10:30:00Z",
)
```
```bash cURL theme={null}
LAMBDADB_TARGET_PROJECT_API_KEY="YOUR_TARGET_API_KEY"
LAMBDADB_SOURCE_PROJECT_API_KEY="YOUR_SOURCE_API_KEY"
curl -i -X POST "$BASE_URL/projects/$PROJECT_NAME/collections" \
-H "content-type: application/json" \
-H "x-api-key: $LAMBDADB_TARGET_PROJECT_API_KEY" \
-d '{
"collectionName": "restored-collection",
"sourceProjectName": "source-project-name",
"sourceProjectApiKey": "'$LAMBDADB_SOURCE_PROJECT_API_KEY'",
"sourceCollectionName": "source-collection-name",
"sourceDatetime": "2024-01-15T10:30:00Z"
}'
```
PITR allows you to restore collections to any point within the configured retention period.
The `sourceDatetime` parameter must be in ISO 8601 format (UTC timezone). If `sourceDatetime` is not specified, the collection will be restored from the most recent data available.
## Fork a collection with additional index configs
You can fork a collection based on an existing collection and extend it with additional indexConfigs. The original collection's configuration will be preserved, and you can only add new fields.
```python Python theme={null}
from lambdadb import LambdaDB, models
with LambdaDB(project_api_key="YOUR_TARGET_API_KEY", base_url="YOUR_BASE_URL", project_name="YOUR_PROJECT_NAME") as client:
forked = client.collections.create(
collection_name="forked-collection",
source_project_name="source-project-name",
source_project_api_key="YOUR_SOURCE_API_KEY",
source_collection_name="example-source-collection-name",
source_datetime="2025-08-08T10:00:00Z",
index_configs={
"newText": {"type": "text", "analyzers": ["japanese", "korean", "english"]},
},
)
```
```bash cURL theme={null}
LAMBDADB_TARGET_PROJECT_API_KEY="YOUR_TARGET_API_KEY"
LAMBDADB_SOURCE_PROJECT_API_KEY="YOUR_SOURCE_API_KEY"
curl -i -X POST "$BASE_URL/projects/$PROJECT_NAME/collections" \
-H "content-type: application/json" \
-H "x-api-key: $LAMBDADB_TARGET_PROJECT_API_KEY" \
-d '{
"collectionName": "forked-collection",
"sourceProjectId": "example-source-project-id",
"sourceProjectApiKey": "'$LAMBDADB_SOURCE_PROJECT_API_KEY'",
"sourceCollectionName": "example-source-collection-name",
"sourceDatetime": "2025-08-08T10:00:00Z"
}'
```
When forking a collection, specifying `indexConfigs` is optional.
You can pass additional `indexConfigs` to extend the original collection's configuration,
but deleting or modifying the original collection's indexConfigs is not allowed.
## Collection limits
| Metric | Limit |
| :--------------------------------- | :-------- |
| Max number of collections | unlimited |
| Max forked children per collection | 30 |
# Index types
Source: https://docs.lambdadb.ai/guides/collections/index-types
Reference for LambdaDB's nine index types — text, keyword, vector, sparse vector, boolean, datetime, long, double, and object — with analyzer options.
LambdaDB currently supports nine types of indexes. Note that the whole document is stored as is regardless of its existence in index configurations, but being stored does not necessarily mean it is searchable.
The dot (.) character cannot be used as a field name, and you can only add indexes to existing collections. Modifying or deleting existing indexes is not supported.
## text
This type is for full-text values, such as the body of an email or the description of a product. These full-text values are analyzed by passing them through an analyzer to convert the string into a list of individual terms before being indexed. The analysis process allows LambdaDB to search for individual words within each full text field.
`text` indexes are best suited for unstructured but human-readable content. If you need to index structured content such as email addresses, hostnames, status codes, or tags, you should rather use a `keyword` index.
LambdaDB supports four analyzers for tokenization: `standard` (default), `korean`, `japanese`, `english`. You can specify multiple analyzers for a single text field to improve search performance.
```python Python theme={null}
{
"content": {
"type": "text",
"analyzers": [
"english",
"korean",
"japanese"
]
}
}
```
```json JSON theme={null}
{
"content": {
"type": "text",
"analyzers": [
"english",
"korean",
"japanese"
]
}
}
```
## keyword
`keyword` type is used for structured content such as IDs, email addresses, hostnames, status codes, zip codes, or tags. Keyword indexes are often used in sorting, aggregations, and term-level queries.
LambdaDB does not index any string longer than 4096 characters.
You can also store multiple keyword values as an array for fields like tags or categories.
```python Python theme={null}
{
"status": {"type": "keyword"},
"category": {"type": "keyword"}
}
```
```json JSON theme={null}
{
"status": {"type": "keyword"},
"category": {"type": "keyword"}
}
```
## long
This type is for a signed 64-bit integer with a minimum value of $-2^{63}$ and a maximum value of $2^{63}-1$. `long` indexes are optimized for scoring, sorting, and range queries.
```python Python theme={null}
{
"user_id": {"type": "long"},
"timestamp": {"type": "long"}
}
```
```json JSON theme={null}
{
"user_id": {"type": "long"},
"timestamp": {"type": "long"}
}
```
## double
This type is for a double-precision 64-bit IEEE 754 floating point number, restricted to finite values. `double` indexes are optimized for scoring, sorting, and range queries.
```python Python theme={null}
{
"score": {"type": "double"},
"price": {"type": "double"}
}
```
```json JSON theme={null}
{
"score": {"type": "double"},
"price": {"type": "double"}
}
```
## boolean
`boolean` indexes accept JSON true and false values, but can also accept strings which are interpreted as either true or false.
```python Python theme={null}
{
"is_active": {"type": "boolean"},
"published": {"type": "boolean"}
}
```
```json JSON theme={null}
{
"is_active": {"type": "boolean"},
"published": {"type": "boolean"}
}
```
## datetime
This type is for date and time in [RFC 3339 format](https://datatracker.ietf.org/doc/html/rfc3339#section-5.6). `datetime` indexes are optimized for sorting and range queries.
```python Python theme={null}
import pytz
from datetime import datetime
# Example datetime usage
dt = datetime.now()
print(dt.astimezone(pytz.UTC).isoformat(timespec="seconds"))
# Output: 2024-11-05T14:27:56+00:00
# Index configuration
{
"created_at": {"type": "datetime"},
"updated_at": {"type": "datetime"}
}
```
```json JSON theme={null}
{
"created_at": {"type": "datetime"},
"updated_at": {"type": "datetime"}
}
```
## vector
The `vector` type indexes dense vectors of numeric values. `vector` indexes are primarily used for k-nearest neighbor (kNN) search. The vector type does not support aggregations or sorting. You add a vector field as an array of numeric values.
A kNN search finds the k nearest vectors to a query vector, as measured by a similarity metric. LambdaDB supports four similarity metrics: `euclidean`, `dot_product`, `cosine`, `max_inner_product`. You can define the vector similarity to use in kNN search.
LambdaDB supports two vector field modes:
* unmanaged vector fields, where you send vector values directly
* managed embedding vector fields, where LambdaDB generates vector values from a source text field
LambdaDB also supports `multi-field vector search`, allowing you to perform kNN searches across multiple vector fields simultaneously within a single query. This enables complex semantic search scenarios where you can combine different types of embeddings (e.g., text embeddings, image embeddings) in one search operation.
```python Python theme={null}
{
"embedding": {
"type": "vector",
"dimensions": 768,
"similarity": "cosine"
},
"image_vector": {
"type": "vector",
"dimensions": 512,
"similarity": "euclidean"
}
}
```
```json JSON theme={null}
{
"embedding": {
"type": "vector",
"dimensions": 768,
"similarity": "cosine"
},
"image_vector": {
"type": "vector",
"dimensions": 512,
"similarity": "euclidean"
}
}
```
### Managed embedding vector fields
Use `managedEmbedding: true` to let LambdaDB generate embeddings for a vector field from a source text field.
For the current provider and model matrix, see [Managed embeddings](/guides/collections/managed-embeddings).
```python Python theme={null}
{
"body": {
"type": "text",
"analyzers": ["english"]
},
"body_embedding": {
"type": "vector",
"managedEmbedding": True,
"embedding": {
"provider": "openai",
"model": "text-embedding-3-small",
"sourceField": "body"
}
}
}
```
```json JSON theme={null}
{
"body": {
"type": "text",
"analyzers": ["english"]
},
"body_embedding": {
"type": "vector",
"managedEmbedding": true,
"embedding": {
"provider": "openai",
"model": "text-embedding-3-small",
"sourceField": "body"
}
}
}
```
For managed embedding vector fields:
* `embedding.provider`, `embedding.model`, and `embedding.sourceField` are required
* `embedding.sourceField` must point to a `text` field in the same collection
* do not send direct vector values for the managed field in upsert or update requests
* use `knn.queryText` for search instead of `knn.queryVector`
* `bulk upsert` is not supported for collections that contain managed embedding fields
## sparseVector
The `sparseVector` type is designed for storing and indexing sparse vectors, where most elements are zero or missing. Unlike dense vectors, sparse vectors only store non-zero values along with their corresponding indexes. `sparseVector` type only supports the `dot_product` distance metric.
```python Python theme={null}
{
"sparse_embedding": {"type": "sparseVector"}
}
```
```json JSON theme={null}
{
"sparse_embedding": {"type": "sparseVector"}
}
```
## object
JSON documents are hierarchical in nature: the document may contain inner objects which, in turn, may contain inner objects themselves. Internally, this document is indexed as a simple, flat list of key-value pairs. The fields within the object can be of any data type, including object. `objectIndexConfigs` should be specified in order to index the fields inside the object.
```python Python theme={null}
{
"metadata": {
"type": "object",
"objectIndexConfigs": {
"url": {"type": "keyword"},
"author": {"type": "keyword"},
"content": {
"type": "text",
"analyzers": ["english", "korean"]
}
}
}
}
```
```json JSON theme={null}
{
"metadata": {
"type": "object",
"objectIndexConfigs": {
"url": {"type": "keyword"},
"author": {"type": "keyword"},
"content": {
"type": "text",
"analyzers": ["english", "korean"]
}
}
}
}
```
## Complete example configuration
Here's a comprehensive example that demonstrates all index types:
```python Python theme={null}
from lambdadb import models
complete_index_config = {
"text": {
"type": "text",
"analyzers": [
"japanese",
"korean",
"english",
],
},
"keyword": {"type": "keyword"},
"long": {"type": "long"},
"double": {"type": "double"},
"boolean": {"type": "boolean"},
"datetime": {"type": "datetime"},
"vector": {
"type": "vector",
"dimensions": 10,
"similarity": "cosine",
},
"sparseVector": {"type": "sparseVector"},
"object": {
"type": "object",
"objectIndexConfigs": {
"text": {
"type": "text",
"analyzers": [
"japanese",
"korean",
"english",
],
},
"keyword": {"type": "keyword"},
"long": {"type": "long"},
"double": {"type": "double"},
"boolean": {"type": "boolean"},
"datetime": {"type": "datetime"},
"vector": {
"type": "vector",
"dimensions": 10,
"similarity": "cosine",
},
"sparseVector": {"type": "sparseVector"},
},
},
}
```
```json JSON theme={null}
{
"text": {
"type": "text",
"analyzers": [
"japanese",
"korean",
"english"
]
},
"keyword": {"type": "keyword"},
"long": {"type": "long"},
"double": {"type": "double"},
"boolean": {"type": "boolean"},
"datetime": {"type": "datetime"},
"vector": {
"type": "vector",
"dimensions": 10,
"similarity": "cosine"
},
"sparseVector": {"type": "sparseVector"},
"object": {
"type": "object",
"objectIndexConfigs": {
"text": {
"type": "text",
"analyzers": [
"japanese",
"korean",
"english"
]
},
"keyword": {"type": "keyword"},
"long": {"type": "long"},
"double": {"type": "double"},
"boolean": {"type": "boolean"},
"datetime": {"type": "datetime"},
"vector": {
"type": "vector",
"dimensions": 10,
"similarity": "cosine"
},
"sparseVector": {"type": "sparseVector"}
}
}
}
```
## Supported analyzers
| Analyzer | Description |
| :--------- | :------------------------------- |
| `standard` | Default general-purpose analyzer |
| `english` | English language analyzer |
| `korean` | Korean language analyzer |
| `japanese` | Japanese language analyzer |
Leave a comment in our [community channel](https://discord.gg/dnKm7WUHWg) or [contact us](https://lambdadb.ai/contact) if you need an analyzer not listed above.
## Supported similarity metrics
| Metric | Description | Use Case |
| :------------------ | :--------------------- | :------------------------------ |
| `cosine` | Cosine similarity | Most common for text embeddings |
| `euclidean` | Euclidean distance | Geometric distance calculations |
| `dot_product` | Dot product similarity | Fast similarity computation |
| `max_inner_product` | Maximum inner product | Specialized similarity metric |
# Manage collections
Source: https://docs.lambdadb.ai/guides/collections/manage-collections
List, describe, and delete LambdaDB collections using the Python, TypeScript, or Go SDKs and the REST API. Includes pagination and code examples.
This page shows you how to manage your existing collections including listing, describing, and deleting operations.
## List collections
Retrieve a complete list of all collections in your project along with their metadata and status information. The endpoint supports optional pagination.
| Parameter | Description | Type | Required | Default |
| :-------- | :----------------------------------------------- | :------ | :------- | :------ |
| size | Max number of collections to return in one page. | integer | | |
| pageToken | Token for the next page of results. | string | | |
```python Python theme={null}
from lambdadb import LambdaDB
with LambdaDB(project_api_key="YOUR_API_KEY", base_url="YOUR_BASE_URL", project_name="YOUR_PROJECT_NAME") as client:
# First page (optional: size, page_token)
res = client.collections.list(size=20)
for c in res.collections:
print(c.collection_name, c.num_docs)
# Next page (if res.next_page_token is set)
if res.next_page_token:
res_next = client.collections.list(size=20, page_token=res.next_page_token)
```
Python: `LambdaDB` supports context manager usage. `__enter__` returns the client, and `__exit__` calls `client.close()` (closing the SDK-owned HTTP client) and makes the client unusable after the `with` block. If you don't use `with`, call `client.close()` when you're done. If you pass a custom `client=`/`async_client=`, you own closing it.
```typescript TypeScript theme={null}
import { LambdaDBClient } from "@functional-systems/lambdadb";
const client = new LambdaDBClient({
projectApiKey: "YOUR_API_KEY",
baseUrl: "YOUR_BASE_URL",
projectName: "YOUR_PROJECT_NAME",
});
// First page (optional: size, pageToken)
const res = await client.listCollections({ size: 20 });
console.log(res.collections);
// Next page (if res.nextPageToken is set)
if (res.nextPageToken) {
const resNext = await client.listCollections({ size: 20, pageToken: res.nextPageToken });
}
```
```go Go theme={null}
client := lambdadb.New(
lambdadb.WithBaseURL("YOUR_BASE_URL"),
lambdadb.WithProjectName("YOUR_PROJECT_NAME"),
lambdadb.WithAPIKey("YOUR_API_KEY"),
)
// First page (optional: Size, PageToken)
opts := &lambdadb.ListCollectionsOpts{Size: lambdadb.Int64(20)}
res, err := client.Collections.List(ctx, opts)
if err != nil { log.Fatal(err) }
for _, c := range res.Collections { _ = c }
// Next page (if res.NextPageToken != nil)
if res.NextPageToken != nil {
resNext, _ := client.Collections.List(ctx, &lambdadb.ListCollectionsOpts{Size: lambdadb.Int64(20), PageToken: res.NextPageToken})
_ = resNext
}
```
```bash cURL theme={null}
# $BASE_URL and $PROJECT_NAME match your LambdaDB Cloud project's region-specific base URL and project name.
LAMBDADB_PROJECT_API_KEY="YOUR_API_KEY"
# First page (optional: size, pageToken)
curl -i -X GET "$BASE_URL/projects/$PROJECT_NAME/collections?size=20" \
-H "x-api-key: $LAMBDADB_PROJECT_API_KEY"
# Next page (use nextPageToken from previous response)
curl -i -X GET "$BASE_URL/projects/$PROJECT_NAME/collections?size=20&pageToken=YOUR_PAGE_TOKEN" \
-H "x-api-key: $LAMBDADB_PROJECT_API_KEY"
```
The response includes detailed information about each collection. When there are more results, the response may include `nextPageToken` for fetching the next page:
```json theme={null}
{
"collections": [
{
"projectName": "your-project",
"collectionName": "example-collection",
"indexConfigs": {
"title": {"type": "text"},
"category": {"type": "keyword"}
},
"numDocs": 1250,
"sourceProjectName": "source-project-name",
"sourceCollectionName": "source-collection-name",
"sourceCollectionVersionId": "wR0NyDJbqDiHMaaV597GjczO2oGQyG7T",
"collectionStatus": "ACTIVE"
}
],
"nextPageToken": "eyJ..."
}
```
## Describe a collection
Get detailed information about a specific collection, including its configuration, document count, and current status.
```python Python theme={null}
with LambdaDB(project_api_key="YOUR_API_KEY", base_url="YOUR_BASE_URL", project_name="YOUR_PROJECT_NAME") as client:
meta = client.collection("example-collection").get()
# or client.collections.get(collection_name="example-collection")
```
```typescript TypeScript theme={null}
const meta = await client.collection("example-collection").get();
console.log(meta);
```
```go Go theme={null}
meta, err := client.Collection("example-collection").Get(ctx)
if err != nil { log.Fatal(err) }
```
```bash cURL theme={null}
curl -i -X GET "$BASE_URL/projects/$PROJECT_NAME/collections/example-collection" \
-H "x-api-key: $LAMBDADB_PROJECT_API_KEY"
```
The response provides comprehensive collection details:
```json theme={null}
{
"projectName": "your-project",
"collectionName": "example-collection",
"indexConfigs": {
"title": {
"type": "text",
"analyzers": ["standard", "korean"]
},
"embedding": {
"type": "vector",
"dimensions": 768,
"similarity": "cosine"
}
},
"numDocs": 1250,
"sourceProjectName": "source-project-name",
"sourceCollectionName": "source-collection-name",
"sourceCollectionVersionId": "wR0NyDJbqDiHMaaV597GjczO2oGQyG7T",
"collectionStatus": "ACTIVE"
}
```
## Delete a collection
Permanently delete a collection and all of its associated data and resources. This operation cannot be undone.
Deletion is **asynchronous**: the API returns **202 Accepted** and queues the work in the background. Until the job finishes, the collection name is still reserved—you may see `collectionStatus` **`DELETING`** from [Describe a collection](#describe-a-collection), and creating a collection with the same name can fail with a resource-already-exists error. For large collections (for example thousands of documents), this phase can take **a minute or longer**. Use the [wait-for-deletion pattern](#wait-for-deletion-before-recreating) before calling create again with the same name.
```python Python theme={null}
client.collections.delete(collection_name="collection-to-delete")
```
```typescript TypeScript theme={null}
await client.collection("collection-to-delete").delete();
```
```go Go theme={null}
err := client.Collection("collection-to-delete").Delete(ctx)
if err != nil { log.Fatal(err) }
```
```bash cURL theme={null}
curl -i -X DELETE "$BASE_URL/projects/$PROJECT_NAME/collections/collection-to-delete" \
-H "x-api-key: $LAMBDADB_PROJECT_API_KEY"
```
Deleting a collection is irreversible and will permanently remove all documents and metadata. Make sure you have proper backups if needed before proceeding with this operation.
## Collection status
Collections can have the following status values:
| Status | Description |
| :--------- | :------------------------------------------------------------------------------------------ |
| `CREATING` | Collection is being created and not yet available |
| `ACTIVE` | Collection is ready for read/write operations |
| `DELETING` | Collection is being deleted asynchronously; the name stays reserved until removal completes |
## Wait for deletion before recreating
If you delete a collection and immediately call create with the **same** `collectionName`, the create request may fail (for example **Resource already exists** / `ResourceAlreadyExists`) even though you already issued delete. That is expected while the collection is still being removed.
**What to do:** After a successful delete call, poll **Describe** (`get`) until the collection **no longer exists** (typically HTTP **404** / `ResourceNotFound` from the API), then create. Checking `collectionStatus === "DELETING"` on each successful `get` confirms you are still in the teardown phase.
**Prefer `get` over `list` for this:** `list` can still include the collection for some time after delete; describe-by-name reflects the lifecycle more directly for a single collection.
```python Python theme={null}
import time
collection_name = "knowledge-layer"
poll_interval_s = 2.0
max_wait_s = 900.0 # allow long-running deletes for large collections
with LambdaDB(project_api_key="YOUR_API_KEY", base_url="YOUR_BASE_URL", project_name="YOUR_PROJECT_NAME") as client:
client.collections.delete(collection_name=collection_name)
deadline = time.monotonic() + max_wait_s
while time.monotonic() < deadline:
try:
meta = client.collections.get(collection_name=collection_name)
except Exception as exc:
# When the collection is fully gone, describe returns 404 — use the not-found type your SDK maps to that response.
code = getattr(exc, "status_code", None) or getattr(exc, "status", None)
if code == 404:
break
raise
# Still present (often with status=DELETING); the name is not free yet.
time.sleep(poll_interval_s)
else:
raise TimeoutError(f"Timed out after {max_wait_s}s waiting for {collection_name!r} to be removed")
# Use the same index_configs you would pass to a normal create.
client.collections.create(collection_name=collection_name, index_configs=your_index_configs)
```
```typescript TypeScript theme={null}
import { LambdaDBClient } from "@functional-systems/lambdadb";
const collectionName = "knowledge-layer";
const pollMs = 2000;
const timeoutMs = 900_000;
const client = new LambdaDBClient({
projectApiKey: "YOUR_API_KEY",
baseUrl: "YOUR_BASE_URL",
projectName: "YOUR_PROJECT_NAME",
});
await client.collection(collectionName).delete();
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
try {
// Still present (often with status=DELETING); the name is not free yet.
await client.collection(collectionName).get();
await new Promise((r) => setTimeout(r, pollMs));
} catch (e: unknown) {
// When the collection is fully removed, describe should fail with a not-found style error (often HTTP 404).
const msg = e instanceof Error ? e.message : String(e);
if (msg.toLowerCase().includes("not found") || msg.includes("404")) break;
throw e;
}
}
if (Date.now() >= deadline) throw new Error(`Timeout waiting for ${collectionName} to be removed`);
await client.createCollection({
collectionName,
indexConfigs: {
/* same fields as a normal create */
},
});
```
```go Go theme={null}
package main
import (
"context"
"errors"
"log"
"time"
lambdadb "github.com/lambdadb/go-lambdadb"
"github.com/lambdadb/go-lambdadb/models/apierrors"
"github.com/lambdadb/go-lambdadb/models/components"
)
func example(ctx context.Context, client *lambdadb.Client) {
poll := 2 * time.Second
deadline := time.Now().Add(900 * time.Second)
name := "knowledge-layer"
if _, err := client.Collection(name).Delete(ctx); err != nil {
log.Fatal(err)
}
for time.Now().Before(deadline) {
meta, err := client.Collection(name).Get(ctx)
if err != nil {
var nf *apierrors.ResourceNotFoundError
if errors.As(err, &nf) {
break
}
log.Fatal(err)
}
_ = meta // still present (often with status=DELETING); the name is not free yet
time.Sleep(poll)
}
if time.Now().After(deadline) {
log.Fatalf("timeout waiting for collection %q to be removed", name)
}
_, err := client.Collections.Create(ctx, lambdadb.CreateCollectionOptions{
CollectionName: name,
IndexConfigs: map[string]interface{}{
/* same fields as a normal create */
},
})
if err != nil {
log.Fatal(err)
}
}
```
# Managed embeddings
Source: https://docs.lambdadb.ai/guides/collections/managed-embeddings
Configure managed embedding vector fields in LambdaDB, including supported providers, supported models, dimension rules, and query behavior.
Managed embeddings let LambdaDB generate vector values from a source text field and store them in a managed `vector` field.
Use managed embeddings when you want LambdaDB to own:
* embedding model selection
* vector generation during document writes
* query embedding generation for vector search
Managed embedding generation consumes **inference usage**. Inference usage is measured in LIU (LambdaDB Inference Unit). See [Understanding costs](/guides/costs/understanding-costs#inference-usage).
## Supported providers
LambdaDB currently supports the following embedding provider:
| Provider | Status |
| :------- | :-------- |
| `openai` | Supported |
## Supported models
The following OpenAI embedding models are currently supported for managed embedding vector fields.
| Model | Default dimensions | Dimensions parameter | Similarity |
| :----------------------- | :----------------- | :--------------------------- | :--------- |
| `text-embedding-3-small` | `1536` | Optional, from `1` to `1536` | `cosine` |
| `text-embedding-3-large` | `3072` | Optional, from `1` to `3072` | `cosine` |
| `text-embedding-ada-002` | `1536` | Fixed at `1536` | `cosine` |
## Collection schema
Define a managed embedding vector field with `managedEmbedding: true` and an `embedding` block.
```json theme={null}
{
"indexConfigs": {
"body": {
"type": "text",
"analyzers": ["english"]
},
"bodyEmbedding": {
"type": "vector",
"managedEmbedding": true,
"embedding": {
"provider": "openai",
"model": "text-embedding-3-small",
"sourceField": "body"
}
}
}
}
```
## Schema rules
* `embedding.provider` is required
* `embedding.model` is required
* `embedding.sourceField` is required
* `embedding.sourceField` must reference a `text` field in the same collection
* managed embedding vector fields must not use top-level `dimensions`
* managed embedding vector fields must not use top-level `similarity`
* LambdaDB resolves and stores the effective `embedding.dimensions` and `embedding.similarity`
## Write behavior
For managed embedding vector fields, send the source text field and let LambdaDB generate the vector value.
Do not send direct vector values for managed embedding fields in:
* [upsert](/guides/documents/upsert-data)
* [update](/guides/documents/update-data)
Example upsert payload:
```json theme={null}
{
"docs": [
{
"id": "doc-1",
"body": "Refunds are available within 7 days of purchase."
}
]
}
```
## Query behavior
For managed embedding vector fields, use `knn.queryText` instead of `knn.queryVector`.
```json theme={null}
{
"query": {
"knn": {
"field": "bodyEmbedding",
"queryText": "refund policy",
"k": 10
}
}
}
```
For full query examples, see [Vector query](/guides/search/vector).
## Bulk upsert
`bulk upsert` is not supported for collections that contain managed embedding vector fields.
Use the regular document write flow instead:
* [Upsert data](/guides/documents/upsert-data)
* [Update data](/guides/documents/update-data)
# Understanding costs
Source: https://docs.lambdadb.ai/guides/costs/understanding-costs
Learn how LambdaDB serverless pricing works across read usage, write usage, storage, and inference usage. Covers per-request minimums, consistency surcharges, and managed embedding usage.
LambdaDB offers a **Free** plan and a **Standard** plan.
There is **no minimum cost** or base fee. If you have no usage, your cost is **\$0**.
On the **Standard** plan, usage is metered in four dimensions: **read usage**, **write usage**, **storage usage**, and **inference usage**. Charges are based on actual measured usage at the rates below.
For the latest pricing details, see [Pricing](https://lambdadb.ai/pricing).
## Plans
LambdaDB offers two plans:
* **Free**: includes up to **1 PB read usage**, **2 GB write usage**, **2 GB storage usage**, and **500K LIU inference usage** per month at **no cost**. After a Free plan limit is reached, additional usage is **restricted**.
* **Standard**: usage is billed based on actual measured **read usage**, **write usage**, **storage usage**, and **inference usage** as described below.
## Read usage
Read usage is billed at **\$5 per read PB** (petabyte) of measured usage.
The following requests contribute to read usage based on **collection size**:
* **Collection query** ([document query](/reference/api/endpoint/document-query))
* **Document fetch** ([document fetch](/reference/api/endpoint/document-fetch))
* **Document list** ([document list](/reference/api/endpoint/document-list))
For these operations, the size of the collection is reflected in read usage.
**Per-request minimum:** each request is counted as at least **1 GB** of read usage.
**Strong consistency:** for **query** and **fetch** requests, if you set `consistentRead` to `true`, an **additional 1 GB** is added to read usage for that request (on top of the per-request minimum and the collection-size basis). See also [Search overview](/guides/search/search-overview) and [Fetch data](/guides/documents/fetch-data).
## Write usage
Write usage is billed at **\$1 per write GB** of measured usage.
**Upsert**, **update**, and **bulk-upsert** ([document upsert](/reference/api/endpoint/document-upsert), [document update](/reference/api/endpoint/document-update), [document bulk upsert](/reference/api/endpoint/document-bulk-upsert)) are measured from the **size in bytes of the document payload** you send in the request body.
**Update** ([document update](/reference/api/endpoint/document-update)) only: each request always runs an internal **document fetch** with strong consistency (equivalent to **fetch** with `consistentRead` set to `true`). That work is billed as **read usage** under [Read usage](#read-usage)—including collection-size basis, the per-request minimum, and the **Strong consistency** rule.
**Per-request minimum** for upsert, update, and bulk-upsert: **50 KB**.
**Delete** ([document delete](/reference/api/endpoint/document-delete)): **100 KB** of write usage is added **per delete request** (fixed amount).
## Storage usage
Storage is billed on **physical storage** at **\$0.33 per GB-month**.
If you use **branching**, logical storage and physical storage can differ; billing follows **physical** storage.
Usage is computed from **hourly averages** of stored data. Those averages are summed into **GB-hours** over the billing period (not a single end-of-month snapshot).
## Inference usage
Inference usage applies when LambdaDB calls a managed model provider on your behalf, such as managed embedding generation during document writes or `knn.queryText` searches on managed embedding fields.
Inference usage is measured in **LIU** (LambdaDB Inference Unit) and billed at **\$1.05 per 1M LIU**.
LambdaDB converts provider/model token usage into LIU using LambdaDB's internal rate card. For example, different embedding models can have different LIU-per-token conversion rates. Usage is aggregated over the billing period before it is converted to the integer LIU quantity used for billing.
Current managed embedding rates:
| Provider | Model | Input LIU per token | Output LIU per token | Approx. cost per 1M input tokens |
| :------- | :----------------------- | ------------------: | -------------------: | -------------------------------: |
| `openai` | `text-embedding-3-small` | 0.02 | 0 | \$0.021 |
| `openai` | `text-embedding-3-large` | 0.13 | 0 | \$0.1365 |
| `openai` | `text-embedding-ada-002` | 0.1 | 0 | \$0.105 |
If the aggregated LIU value is positive but less than 1, it is rounded up to **1 LIU** so non-zero managed model usage is not dropped.
See also [Managed embeddings](/guides/collections/managed-embeddings).
## Billing cycle
On the **first day of each calendar month**, usage for the **previous month**—read, write, storage, and inference—is aggregated and invoiced for the **Standard** plan.
# Bulk upsert data
Source: https://docs.lambdadb.ai/guides/documents/bulk-upsert-data
Upload up to 200 MB of documents at once using the LambdaDB bulk upsert workflow with presigned URLs. Includes SDK and API code examples.
Besides the upsert operation which has 6MB maximum payload size limit,
LambdaDB also supports bulk upsert operation to insert or update multiple documents up to 200MB at once.
Bulk upsert is not supported for collections that contain managed embedding vector fields. Use the regular [upsert](/guides/documents/upsert-data) or [update](/guides/documents/update-data) flow so LambdaDB can generate embeddings from the configured source text fields.
## Recommended: One-step bulk upsert
The easiest way to bulk upsert is to use the SDK's one-step method. The client uploads your documents to the presigned URL and completes the bulk upsert for you (up to 200MB).
```python Python theme={null}
from lambdadb import LambdaDB
with LambdaDB(
project_api_key="YOUR_API_KEY",
base_url="YOUR_BASE_URL",
project_name="YOUR_PROJECT_NAME",
) as client:
coll = client.collection("my_collection")
docs = [
{"id": "bulk_1", "url": "https://en.wikipedia.org/wiki/LambdaDB", "title": "LambdaDB", "text": "LambdaDB is an AI-native database ... ", "dense_vector": [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0], "sparse_vector": {"LambdaDB": 0.83, "is": 0.1, "a": 0.1, "AI": 0.7}},
{"id": "bulk_2", "url": "https://en.wikipedia.org/wiki/Winamp", "title": "Winamp", "text": "Winamp is a media player for Windows, macOS and Android ...", "dense_vector": [1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 2.0]},
]
coll.docs.bulk_upsert_docs(docs=docs)
```
Python: `LambdaDB` supports context manager usage. `__enter__` returns the client, and `__exit__` calls `client.close()` (closing the SDK-owned HTTP client) and makes the client unusable after the `with` block. If you don't use `with`, call `client.close()` when you're done. If you pass a custom `client=`/`async_client=`, you own closing it.
```typescript TypeScript theme={null}
import { LambdaDBClient } from "@functional-systems/lambdadb";
const client = new LambdaDBClient({
projectApiKey: "YOUR_API_KEY",
baseUrl: "YOUR_BASE_URL",
projectName: "YOUR_PROJECT_NAME",
});
const collection = client.collection("my_collection");
const docs = [
{ id: "bulk_1", url: "https://en.wikipedia.org/wiki/LambdaDB", title: "LambdaDB", text: "LambdaDB is an AI-native database ... ", denseVector: [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0], sparseVector: { LambdaDB: 0.83, is: 0.1, a: 0.1, AI: 0.7 } },
{ id: "bulk_2", url: "https://en.wikipedia.org/wiki/Winamp", title: "Winamp", text: "Winamp is a media player for Windows, macOS and Android ...", denseVector: [1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 2.0] },
];
await collection.docs.bulkUpsertDocs({ docs });
```
```go Go theme={null}
package main
import (
"context"
"log"
lambdadb "github.com/lambdadb/go-lambdadb"
)
func main() {
ctx := context.Background()
client := lambdadb.New(
lambdadb.WithBaseURL("YOUR_BASE_URL"),
lambdadb.WithProjectName("YOUR_PROJECT_NAME"),
lambdadb.WithAPIKey("YOUR_API_KEY"),
)
coll := client.Collection("my_collection")
docs := []map[string]interface{}{
{"id": "bulk_1", "url": "https://en.wikipedia.org/wiki/LambdaDB", "title": "LambdaDB", "text": "LambdaDB is an AI-native database ... ", "dense_vector": []float64{0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0}},
{"id": "bulk_2", "url": "https://en.wikipedia.org/wiki/Winamp", "title": "Winamp", "text": "Winamp is a media player for Windows, macOS and Android ...", "dense_vector": []float64{1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 2.0}},
}
_, err := coll.Docs().BulkUpsertDocuments(ctx, docs)
if err != nil {
log.Fatal(err)
}
}
```
After a successful request you'll receive a message such as `"Upsert request is accepted"`. Documents are processed asynchronously and become searchable after indexing completes.
***
## Two-step: Get presigned URL, then upload and complete
If you need to control the upload yourself (e.g. from a different process or storage), use this flow: (1) get bulk upsert info (presigned URL and `objectKey`), (2) upload the payload to the presigned URL, then (3) call the bulk-upsert API with the `objectKey`.
### Step 1: Get bulk upsert information
```python Python theme={null}
from lambdadb import LambdaDB
with LambdaDB(
project_api_key="YOUR_API_KEY",
base_url="YOUR_BASE_URL",
project_name="YOUR_PROJECT_NAME",
) as client:
coll = client.collection("my_collection")
get_bulk_upsert = coll.docs.get_bulk_upsert()
# get_bulk_upsert.url, get_bulk_upsert.object_key, get_bulk_upsert.size_limit_bytes
```
```typescript TypeScript theme={null}
import { LambdaDBClient } from "@functional-systems/lambdadb";
const client = new LambdaDBClient({
projectApiKey: "YOUR_API_KEY",
baseUrl: "YOUR_BASE_URL",
projectName: "YOUR_PROJECT_NAME",
});
const info = await client.collection("my_collection").docs.getBulkUpsert();
// info.url, info.objectKey, info.sizeLimitBytes
```
```go Go theme={null}
info, err := client.Collection("my_collection").Docs().GetBulkUpsertInfo(ctx)
if err != nil {
log.Fatal(err)
}
// info.URL, info.ObjectKey, info.SizeLimitBytes
```
```bash cURL theme={null}
curl -i -X GET \
"$BASE_URL/projects/$PROJECT_NAME/collections/{collectionName}/docs/bulk-upsert" \
-H 'x-api-key: $LAMBDADB_PROJECT_API_KEY'
```
Response:
```json theme={null}
{
"url": "",
"type": "application/json",
"httpMethod": "PUT",
"objectKey": "",
"sizeLimitBytes": 209715200
}
```
### Step 2: Upload to presigned URL and call bulk-upsert
Upload the document list as JSON to the `url` (PUT), then call the bulk-upsert API with the `objectKey` from step 1.
```python Python theme={null}
import json
import requests
docs = [
{"id": "bulk_33201222", "url": "https://en.wikipedia.org/wiki/LambdaDB", "title": "LambdaDB", "text": "LambdaDB is an AI-native database ... ", "dense_vector": [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0], "sparse_vector": {"LambdaDB": 0.83, "is": 0.1, "a": 0.1, "AI": 0.7}},
]
# Upload to presigned URL
resp = requests.put(
get_bulk_upsert.url,
data=json.dumps({"docs": docs}),
headers={"Content-Type": "application/json"},
)
resp.raise_for_status()
# Complete bulk upsert
coll.docs.bulk_upsert(object_key=get_bulk_upsert.object_key)
```
```typescript TypeScript theme={null}
const docs = [
{ id: "bulk_33201222", url: "https://en.wikipedia.org/wiki/LambdaDB", title: "LambdaDB", text: "LambdaDB is an AI-native database ... ", denseVector: [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0], sparseVector: { LambdaDB: 0.83, is: 0.1, a: 0.1, AI: 0.7 } },
];
// Upload to presigned URL
const uploadRes = await fetch(info.url, {
method: "PUT",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ docs }),
});
if (!uploadRes.ok) throw new Error(`Upload failed: ${uploadRes.status}`);
// Complete bulk upsert
await client.collection("my_collection").docs.bulkUpsert({ objectKey: info.objectKey });
```
```go Go theme={null}
// Assume bytes, encoding/json, net/http are imported
docs := []map[string]interface{}{
{"id": "bulk_33201222", "url": "https://en.wikipedia.org/wiki/LambdaDB", "title": "LambdaDB", "text": "LambdaDB is an AI-native database ... ", "dense_vector": []float64{0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0}},
}
payload, _ := json.Marshal(map[string]interface{}{"docs": docs})
req, _ := http.NewRequestWithContext(ctx, "PUT", info.URL, bytes.NewReader(payload))
req.Header.Set("Content-Type", "application/json")
uploadRes, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatal(err)
}
uploadRes.Body.Close()
if uploadRes.StatusCode < 200 || uploadRes.StatusCode >= 300 {
log.Fatalf("upload failed: %s", uploadRes.Status)
}
// Complete bulk upsert
_, err = client.Collection("my_collection").Docs().BulkUpsert(ctx, lambdadb.BulkUpsertDocsInput{ObjectKey: info.ObjectKey})
if err != nil {
log.Fatal(err)
}
```
```bash cURL theme={null}
# 1. Upload to the presigned URL from step 1
curl -X PUT "" \
-H 'Content-Type: application/json' \
-d '{"docs": [{"id": "bulk_33201222", "url": "https://en.wikipedia.org/wiki/LambdaDB", "title": "LambdaDB", "text": "LambdaDB is an AI-native database ... ", "dense_vector": [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]}]}'
# 2. Complete bulk upsert with the objectKey from step 1
curl -i -X POST \
"$BASE_URL/projects/$PROJECT_NAME/collections/{collectionName}/docs/bulk-upsert" \
-H 'content-type: application/json' \
-H 'x-api-key: $LAMBDADB_PROJECT_API_KEY' \
-d '{"objectKey": ""}'
```
## Response
After successful bulk upsert initiation you'll receive:
```json theme={null}
{
"message": "Bulk upsert request is accepted"
}
```
Bulk upsert operations are processed asynchronously in the background.
Consequently, newly uploaded documents may not be immediately available for search or fetch requests,
even if consistentRead (Python: consistent\_read) is set to true.
The documents will become available only after the indexing process is fully complete.
# Delete data
Source: https://docs.lambdadb.ai/guides/documents/delete-data
Remove documents from a LambdaDB collection by specifying document IDs. Includes Python, TypeScript, Go, and cURL deletion code examples.
This page shows you how to use the delete endpoint to remove documents from a collection.
## Delete documents by IDs
Since LambdaDB documents can always be efficiently accessed using their ID,
deleting by ID is the most efficient way to remove specific records.
```python Python theme={null}
from lambdadb import LambdaDB
with LambdaDB(
project_api_key="YOUR_API_KEY",
base_url="YOUR_BASE_URL",
project_name="YOUR_PROJECT_NAME",
) as client:
coll = client.collection("your_collection_name")
coll.docs.delete(ids=["example-doc-id-1", "example-doc-id-2"])
```
Python: `LambdaDB` supports context manager usage. `__enter__` returns the client, and `__exit__` calls `client.close()` (closing the SDK-owned HTTP client) and makes the client unusable after the `with` block. If you don't use `with`, call `client.close()` when you're done. If you pass a custom `client=`/`async_client=`, you own closing it.
```typescript TypeScript theme={null}
import { LambdaDBClient } from "@functional-systems/lambdadb";
const client = new LambdaDBClient({
projectApiKey: "YOUR_API_KEY",
baseUrl: "YOUR_BASE_URL",
projectName: "YOUR_PROJECT_NAME",
});
await client.collection("your_collection_name").docs.delete({
ids: ["example-doc-id-1", "example-doc-id-2"],
});
```
```go Go theme={null}
package main
import (
"context"
"log"
lambdadb "github.com/lambdadb/go-lambdadb"
)
func main() {
ctx := context.Background()
client := lambdadb.New(
lambdadb.WithBaseURL("YOUR_BASE_URL"),
lambdadb.WithProjectName("YOUR_PROJECT_NAME"),
lambdadb.WithAPIKey("YOUR_API_KEY"),
)
_, err := client.Collection("your_collection_name").Docs().Delete(ctx, lambdadb.DeleteDocsInput{
Ids: []string{"example-doc-id-1", "example-doc-id-2"},
})
if err != nil {
log.Fatal(err)
}
}
```
```bash cURL theme={null}
curl -i -X POST \
--url "$BASE_URL/projects/$PROJECT_NAME/collections/{collectionName}/docs/delete" \
--header 'content-type: application/json' \
--header 'x-api-key: ' \
--data '{
"ids": [
"example-doc-id-1",
"example-doc-id-2"
]
}'
```
## Delete documents by query
To delete documents based on their data, pass a `filter` query to the delete operation.
This deletes all documents matching the filter query.
For example, to delete all documents with genre "documentary" and year 2019 from a collection, use the following code:
```python Python theme={null}
with LambdaDB(
project_api_key="YOUR_API_KEY",
base_url="YOUR_BASE_URL",
project_name="YOUR_PROJECT_NAME",
) as client:
coll = client.collection("your_collection_name")
coll.docs.delete(query_filter={"queryString": {"query": "genre:documentary AND year:2019"}})
```
```typescript TypeScript theme={null}
await client.collection("your_collection_name").docs.delete({
filter: { queryString: { query: "genre:documentary AND year:2019" } },
});
```
```go Go theme={null}
filter := &lambdadb.QueryFilter{QueryString: &lambdadb.QueryStringFilter{Query: "genre:documentary AND year:2019"}}
_, err := client.Collection("your_collection_name").Docs().Delete(ctx, lambdadb.DeleteDocsInput{Filter: filter})
```
```bash cURL theme={null}
curl -i -X POST \
--url "$BASE_URL/projects/$PROJECT_NAME/collections/{collectionName}/docs/delete" \
--header 'content-type: application/json' \
--header 'x-api-key: ' \
--data '{
"filter": {
"queryString": {
"query": "genre:documentary AND year:2019"
}
}
}'
```
## Delete an entire collection
To remove all documents from a collection,
[delete the collection](/guides/collections/manage-collections#delete-a-collection) and
[recreate it](/guides/collections/create-a-collection#create-a-collection-from-scratch).
Deletion is asynchronous: if you reuse the **same** collection name, wait until the name is fully released (for example by polling describe until the collection returns not found) before calling create—see [Wait for deletion before recreating](/guides/collections/manage-collections#wait-for-deletion-before-recreating).
# Fetch data
Source: https://docs.lambdadb.ai/guides/documents/fetch-data
Retrieve documents by ID from a LambdaDB collection. Supports consistent reads, vector inclusion, field filtering, and partition filters.
This page shows you how to use the fetch endpoint to fetch documents by IDs from a collection.
| Parameter | Description | Type | Required | Default |
| :-------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-------- | :------- | :------ |
| ids | The document IDs to fetch up to 100. | string\[] | ✓ | |
| includeVectors | Indicates whether vector values are included in the response. (Python: `include_vectors`) | boolean | | false |
| consistentRead | Determines the read consistency model: If set to true, then the operation uses strongly consistent reads; otherwise, the operation uses eventually consistent reads. (Python: `consistent_read`) | boolean | | false |
| fields | A list of field names to include and/or exclude in the result. Use dot notation (e.g., user.name) to specify nested fields. | object | | |
| partitionFilter | Partition filter. | object | | |
`include` is applied first, and then `exclude` is applied to the included fields when you set both in the `fields` parameter.
To fetch documents, specify the document IDs (up to 100 IDs).
```python Python theme={null}
from lambdadb import LambdaDB
with LambdaDB(
project_api_key="YOUR_API_KEY",
base_url="YOUR_BASE_URL",
project_name="YOUR_PROJECT_NAME",
) as client:
coll = client.collection("my_collection")
res = coll.docs.fetch(
ids=["33201222"],
include_vectors=True,
fields={"include": ["url", "title", "text"], "exclude": ["metadata.raw"]},
)
# `res.docs` contains items (each item includes `doc` and metadata).
# `res.documents` contains document bodies only.
```
Python: `LambdaDB` supports context manager usage. `__enter__` returns the client, and `__exit__` calls `client.close()` (closing the SDK-owned HTTP client) and makes the client unusable after the `with` block. If you don't use `with`, call `client.close()` when you're done. If you pass a custom `client=`/`async_client=`, you own closing it.
```typescript TypeScript theme={null}
import { LambdaDBClient } from "@functional-systems/lambdadb";
const client = new LambdaDBClient({
projectApiKey: "YOUR_API_KEY",
baseUrl: "YOUR_BASE_URL",
projectName: "YOUR_PROJECT_NAME",
});
const res = await client.collection("my_collection").docs.fetch({
ids: ["33201222"],
includeVectors: true,
fields: { include: ["url", "title", "text"], exclude: ["metadata.raw"] },
});
```
```go Go theme={null}
package main
import (
"context"
"log"
lambdadb "github.com/lambdadb/go-lambdadb"
)
func main() {
ctx := context.Background()
client := lambdadb.New(
lambdadb.WithBaseURL("YOUR_BASE_URL"),
lambdadb.WithProjectName("YOUR_PROJECT_NAME"),
lambdadb.WithAPIKey("YOUR_API_KEY"),
)
res, err := client.Collection("my_collection").Docs().Fetch(ctx, lambdadb.FetchDocsInput{
Ids: []string{"33201222"},
// Optional: IncludeVectors, Fields (include/exclude)
})
if err != nil {
log.Fatal(err)
}
_ = res
}
```
```bash cURL theme={null}
curl -X POST "$BASE_URL/projects/$PROJECT_NAME/collections/${collection_name}/fetch" \
-H "Content-Type: application/json" \
-H "x-api-key: ${YOUR_API_KEY}" \
-d '{
"ids": ["33201222"],
"includeVectors": true,
"fields": {
"include": ["url", "title", "text"]
}
}'
```
The response will look like this:
```json theme={null}
{
"took": 76,
"total": 1,
"docs": [
{
"collection": "example_collection",
"doc": {
"id": "33201222",
"url": "https://en.wikipedia.org/wiki/LambdaDB",
"title": "LambdaDB",
"text": "LambdaDB is an AI-native database ... ",
"vector": [0.6, -0.12, 0.65, 0.2, 0.3, ...]
}
}
],
"isDocsInline": true
}
```
The order of the returned documents is not guaranteed to match the order of the IDs in the request.
# List data
Source: https://docs.lambdadb.ai/guides/documents/list-data
Paginate through documents in a LambdaDB collection using the list endpoint. Useful for scanning, exporting, filtering, or iterating over stored records.
This page shows you how to use the list endpoint to list documents in a collection. Use this when you need to iterate over documents (for example, to export or scan) without running a ranked search.
| Parameter | Description | Type | Required | Default |
| :------------- | :------------------------------------------------------------ | :------ | :------- | :------ |
| size | Max number of documents to return per page (1–100). | integer | | 100 |
| pageToken | Token for the next page of results. | string | | |
| includeVectors | Indicates whether vector values are included in the response. | boolean | | false |
```python Python theme={null}
from lambdadb import LambdaDB
with LambdaDB(
project_api_key="YOUR_API_KEY",
base_url="YOUR_BASE_URL",
project_name="YOUR_PROJECT_NAME",
) as client:
coll = client.collection("my_collection")
# List first page of documents
res = coll.docs.list(size=10)
# `res.docs` contains items (each item includes `doc` and metadata).
# `res.documents` contains document bodies only.
for item in res.docs:
print(item)
# List next page using the token from the previous response
if res.next_page_token:
res_next = coll.docs.list(size=10, page_token=res.next_page_token)
```
Python: `LambdaDB` supports context manager usage. `__enter__` returns the client, and `__exit__` calls `client.close()` (closing the SDK-owned HTTP client) and makes the client unusable after the `with` block. If you don't use `with`, call `client.close()` when you're done. If you pass a custom `client=`/`async_client=`, you own closing it.
```typescript TypeScript theme={null}
import { LambdaDBClient } from "@functional-systems/lambdadb";
const client = new LambdaDBClient({
projectApiKey: "YOUR_API_KEY",
baseUrl: "YOUR_BASE_URL",
projectName: "YOUR_PROJECT_NAME",
});
const collection = client.collection("my_collection");
// List first page
const res = await collection.docs.list({ size: 20 });
console.log(res.docs?.length, res.total);
// List next page
if (res.nextPageToken) {
const resNext = await collection.docs.list({
size: 20,
pageToken: res.nextPageToken,
});
}
```
```go Go theme={null}
package main
import (
"context"
"log"
lambdadb "github.com/lambdadb/go-lambdadb"
)
func main() {
ctx := context.Background()
client := lambdadb.New(
lambdadb.WithBaseURL("YOUR_BASE_URL"),
lambdadb.WithProjectName("YOUR_PROJECT_NAME"),
lambdadb.WithAPIKey("YOUR_API_KEY"),
)
coll := client.Collection("my_collection")
// List first page
res, err := coll.Docs().List(ctx, &lambdadb.ListDocsOpts{Size: lambdadb.Int64(10)})
if err != nil {
log.Fatal(err)
}
for _, d := range res.Docs {
_ = d
}
// List next page
if res.NextPageToken != nil {
resNext, _ := coll.Docs().List(ctx, &lambdadb.ListDocsOpts{
Size: lambdadb.Int64(10),
PageToken: res.NextPageToken,
})
_ = resNext
}
}
```
```bash cURL theme={null}
# List first page ($BASE_URL and $PROJECT_NAME match SDK base_url / project_name)
curl -X GET "$BASE_URL/projects/$PROJECT_NAME/collections/${collection_name}/docs?size=10" \
-H "x-api-key: ${YOUR_API_KEY}"
# Include vector values when needed
curl -X GET "$BASE_URL/projects/$PROJECT_NAME/collections/${collection_name}/docs?size=10&includeVectors=true" \
-H "x-api-key: ${YOUR_API_KEY}"
# List next page (use nextPageToken from previous response)
curl -X GET "$BASE_URL/projects/$PROJECT_NAME/collections/${collection_name}/docs?size=10&pageToken=YOUR_PAGE_TOKEN" \
-H "x-api-key: ${YOUR_API_KEY}"
```
The response includes the total count, the current page of documents, and an optional token for the next page:
```json theme={null}
{
"total": 2,
"docs": [
{
"collection": "example-collection-name",
"doc": {
"id": "doc-1",
"title": "Example document 1",
"category": "docs"
}
},
{
"collection": "example-collection-name",
"doc": {
"id": "doc-2",
"title": "Example document 2",
"category": "docs"
}
}
],
"nextPageToken": "eyJpZCI6ICJhYmMiLCAiY3JlYXRlZF9hdCI6ICIyMDI0LTExLTA1VDE0OjI3OjU2KzAwOjAwIn0=",
"isDocsInline": true
}
```
## Extended list request
For filtering, partition filtering, field selection, or a JSON request body, use the extended list endpoint:
```python Python theme={null}
from lambdadb import LambdaDB
with LambdaDB(
project_api_key="YOUR_API_KEY",
base_url="YOUR_BASE_URL",
project_name="YOUR_PROJECT_NAME",
) as client:
coll = client.collection("my_collection")
res = coll.docs.list(
size=10,
filter_={"queryString": {"query": "category:docs"}},
partition_filter={"field": "tenant", "in": ["acme"]},
fields={"include": ["id", "title", "category"]},
include_vectors=False,
)
```
```typescript TypeScript theme={null}
import { LambdaDBClient } from "@functional-systems/lambdadb";
const client = new LambdaDBClient({
projectApiKey: "YOUR_API_KEY",
baseUrl: "YOUR_BASE_URL",
projectName: "YOUR_PROJECT_NAME",
});
const res = await client.collection("my_collection").docs.list({
size: 10,
filter: { queryString: { query: "category:docs" } },
partitionFilter: { field: "tenant", in: ["acme"] },
fields: { include: ["id", "title", "category"] },
includeVectors: false,
});
```
```go Go theme={null}
package main
import (
"context"
"log"
lambdadb "github.com/lambdadb/go-lambdadb"
"github.com/lambdadb/go-lambdadb/models/components"
)
func main() {
ctx := context.Background()
client := lambdadb.New(
lambdadb.WithBaseURL("YOUR_BASE_URL"),
lambdadb.WithProjectName("YOUR_PROJECT_NAME"),
lambdadb.WithAPIKey("YOUR_API_KEY"),
)
fields := components.CreateFieldsSelectorUnionFieldsSelector1(components.FieldsSelector1{
Include: []string{"id", "title", "category"},
})
res, err := client.Collection("my_collection").Docs().List(ctx, &lambdadb.ListDocsOpts{
Size: lambdadb.Int64(10),
Filter: map[string]any{
"queryString": map[string]any{"query": "category:docs"},
},
PartitionFilter: &components.PartitionFilter{
Field: "tenant",
In: []string{"acme"},
},
Fields: &fields,
IncludeVectors: lambdadb.Bool(false),
})
if err != nil {
log.Fatal(err)
}
_ = res
}
```
```bash cURL theme={null}
curl -X POST "$BASE_URL/projects/$PROJECT_NAME/collections/${collection_name}/docs/list" \
-H "Content-Type: application/json" \
-H "x-api-key: ${YOUR_API_KEY}" \
--data '{
"size": 10,
"filter": {
"queryString": {
"query": "category:docs"
}
},
"partitionFilter": {
"field": "tenant",
"in": ["acme"]
},
"fields": {
"include": ["id", "title", "category"]
},
"includeVectors": false
}'
```
The SDK scoped list helpers (`docs.list` in Python and TypeScript, `Docs().List` in Go) use the extended list request automatically when you pass filter, partition filter, or field-selection options.
In Go, pass these as `Filter`, `PartitionFilter`, `Fields`, and `IncludeVectors` on `lambdadb.ListDocsOpts`.
The extended list request body supports:
| Parameter | Description | Type | Required | Default |
| :-------------- | :---------------------------------------------------------------------------------- | :------ | :------- | :------ |
| size | Max number of documents to return per page (1–100). | integer | | 100 |
| pageToken | Token for the next page of results. | string | | |
| filter | Filter applied before pagination. Supports `queryString` and nested `bool` filters. | object | | |
| partitionFilter | Restricts the request to matching partition values. | object | | |
| fields | Field selector with `include` and/or `exclude`. Use dot notation for nested fields. | object | | |
| includeVectors | Indicates whether vector values are included in the response. | boolean | | false |
`filter` uses the same `queryString` and `bool` shapes as the query API. List filters do not run vector or ranking queries.
Use `partitionFilter` only for collections configured with a partition field.
Inside a `bool` filter, `occur` can be `filter`, `must`, `must_not`, or `should`; when omitted, it defaults to `should`.
```json theme={null}
{
"filter": {
"bool": [
{
"queryString": {
"query": "category:docs"
},
"occur": "filter"
},
{
"queryString": {
"query": "title:LambdaDB"
},
"occur": "must"
}
]
}
}
```
When you use `nextPageToken` with the extended endpoint, keep the same `filter`, `partitionFilter`, `fields`, and `includeVectors` options across pages.
To fetch specific documents by ID, use the [fetch](/guides/documents/fetch-data) endpoint. To run ranked text, vector, sparse-vector, or hybrid search, use the [query](/guides/search/search-overview) API.
# Update data
Source: https://docs.lambdadb.ai/guides/documents/update-data
Update existing documents in a LambdaDB collection by ID. Modify specific fields without replacing the entire document using SDK or REST API calls.
This page shows you how to update documents in a collection.
If the collection contains managed embedding vector fields, update the configured source text fields instead of sending direct vector values for managed embedding fields.
```python Python theme={null}
from lambdadb import LambdaDB
with LambdaDB(
project_api_key="YOUR_API_KEY",
base_url="YOUR_BASE_URL",
project_name="YOUR_PROJECT_NAME",
) as client:
coll = client.collection("my_collection")
docs = [
{"id": "33201222", "title": "LambdaDB", "text": "LambdaDB is an AI-native database ... "},
{"id": "33201223", "url": "https://en.wikipedia.org/wiki/Winamp", "title": "Winamp v2", "text": "Winamp v2 is a media player for Windows, macOS and Android, originally developed by Nullsoft."},
]
coll.docs.update(docs=docs)
```
Python: `LambdaDB` supports context manager usage. `__enter__` returns the client, and `__exit__` calls `client.close()` (closing the SDK-owned HTTP client) and makes the client unusable after the `with` block. If you don't use `with`, call `client.close()` when you're done. If you pass a custom `client=`/`async_client=`, you own closing it.
```typescript TypeScript theme={null}
import { LambdaDBClient } from "@functional-systems/lambdadb";
const client = new LambdaDBClient({
projectApiKey: "YOUR_API_KEY",
baseUrl: "YOUR_BASE_URL",
projectName: "YOUR_PROJECT_NAME",
});
await client.collection("my_collection").docs.update({
docs: [
{ id: "33201222", title: "LambdaDB", text: "LambdaDB is an AI-native database ... " },
{ id: "33201223", url: "https://en.wikipedia.org/wiki/Winamp", title: "Winamp v2", text: "Winamp v2 is a media player for Windows, macOS and Android." },
],
});
```
```go Go theme={null}
package main
import (
"context"
"log"
lambdadb "github.com/lambdadb/go-lambdadb"
)
func main() {
ctx := context.Background()
client := lambdadb.New(
lambdadb.WithBaseURL("YOUR_BASE_URL"),
lambdadb.WithProjectName("YOUR_PROJECT_NAME"),
lambdadb.WithAPIKey("YOUR_API_KEY"),
)
docs := []map[string]interface{}{
{"id": "33201222", "title": "LambdaDB", "text": "LambdaDB is an AI-native database ... "},
{"id": "33201223", "title": "Winamp v2", "text": "Winamp v2 is a media player for Windows, macOS and Android."},
}
_, err := client.Collection("my_collection").Docs().Update(ctx, lambdadb.UpdateDocsInput{Docs: docs})
if err != nil {
log.Fatal(err)
}
}
```
```bash cURL theme={null}
LAMBDADB_PROJECT_API_KEY="YOUR_API_KEY"
curl -X POST \
"$BASE_URL/projects/$PROJECT_NAME/collections/{collectionName}/docs/update" \
-H 'content-type: application/json' \
-H 'x-api-key: $LAMBDADB_PROJECT_API_KEY' \
-d '{
"docs": [
{
"id": "33201222",
"title": "LambdaDB",
"text": "LambdaDB is an AI-native database ... "
},
{
"id": "33201223",
"url": "https://en.wikipedia.org/wiki/Winamp",
"title": "Winamp v2",
"text": "Winamp v2 is a media player for Windows, macOS and Android, originally developed by Justin Frankel and Dmitry Boldyrev by their company Nullsoft, which they later sold to AOL, who sold to Radionomy in January 2014."
}
]
}'
```
Each document in a payload must contain an `id` field to uniquely identify a document to update.
For a partitioned collection, a document in the `__default__` partition with a matching `id` value is updated if the partition field is not provided.
## Update limits
| Metric | Limit |
| :--------------------------- | :------------- |
| Max payload size | 6MB |
| Max length for a document ID | 512 characters |
| Max vector dimensions | 4,096 |
| Max document size | 5MB |
# Upsert data
Source: https://docs.lambdadb.ai/guides/documents/upsert-data
Insert or update documents in a LambdaDB collection using the upsert operation. Includes Python, TypeScript, Go, and cURL code examples.
This page shows you how to upsert documents into a collection.
If the collection contains managed embedding vector fields, send only the configured source text fields. Do not send direct vector values for managed embedding fields in an upsert request.
```python Python theme={null}
from lambdadb import LambdaDB
with LambdaDB(
project_api_key="YOUR_API_KEY",
base_url="YOUR_BASE_URL",
project_name="YOUR_PROJECT_NAME",
) as client:
coll = client.collection("my_collection")
docs = [
{"id": "33201222", "url": "https://en.wikipedia.org/wiki/LambdaDB", "title": "LambdaDB", "text": "LambdaDB is an AI-native database ... ", "dense_vector": [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0], "sparse_vector": {"LambdaDB": 0.83, "is": 0.1, "a": 0.1, "AI": 0.7}},
{"url": "https://en.wikipedia.org/wiki/Winamp", "title": "Winamp", "text": "Winamp is a media player for Windows, macOS and Android ...", "dense_vector": [1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 2.0], "sparse_vector": {"0": 0.8, "4": 0.1, "11": 0.1, "63": 0.54}},
]
coll.docs.upsert(docs=docs)
```
Python: `LambdaDB` supports context manager usage. `__enter__` returns the client, and `__exit__` calls `client.close()` (closing the SDK-owned HTTP client) and makes the client unusable after the `with` block. If you don't use `with`, call `client.close()` when you're done. If you pass a custom `client=`/`async_client=`, you own closing it.
```typescript TypeScript theme={null}
import { LambdaDBClient } from "@functional-systems/lambdadb";
const client = new LambdaDBClient({
projectApiKey: "YOUR_API_KEY",
baseUrl: "YOUR_BASE_URL",
projectName: "YOUR_PROJECT_NAME",
});
const collection = client.collection("my_collection");
await collection.docs.upsert({
docs: [
{ id: "33201222", url: "https://en.wikipedia.org/wiki/LambdaDB", title: "LambdaDB", text: "LambdaDB is an AI-native database ... ", denseVector: [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0], sparseVector: { LambdaDB: 0.83, is: 0.1, a: 0.1, AI: 0.7 } },
{ url: "https://en.wikipedia.org/wiki/Winamp", title: "Winamp", text: "Winamp is a media player for Windows, macOS and Android ...", denseVector: [1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 2.0], sparseVector: { "0": 0.8, "4": 0.1, "11": 0.1, "63": 0.54 } },
],
});
```
```go Go theme={null}
package main
import (
"context"
"log"
lambdadb "github.com/lambdadb/go-lambdadb"
)
func main() {
ctx := context.Background()
client := lambdadb.New(
lambdadb.WithBaseURL("YOUR_BASE_URL"),
lambdadb.WithProjectName("YOUR_PROJECT_NAME"),
lambdadb.WithAPIKey("YOUR_API_KEY"),
)
coll := client.Collection("my_collection")
myDocs := []map[string]interface{}{
{"id": "33201222", "url": "https://en.wikipedia.org/wiki/LambdaDB", "title": "LambdaDB", "text": "LambdaDB is an AI-native database ... ", "dense_vector": []float64{0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0}, "sparse_vector": map[string]float64{"LambdaDB": 0.83, "is": 0.1, "a": 0.1, "AI": 0.7}},
{"url": "https://en.wikipedia.org/wiki/Winamp", "title": "Winamp", "text": "Winamp is a media player for Windows, macOS and Android ...", "dense_vector": []float64{1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 2.0}},
}
_, err := coll.Docs().Upsert(ctx, lambdadb.UpsertDocsInput{Docs: myDocs})
if err != nil {
log.Fatal(err)
}
}
```
```bash cURL theme={null}
LAMBDADB_PROJECT_API_KEY="YOUR_API_KEY"
curl -X POST \
"$BASE_URL/projects/$PROJECT_NAME/collections/{collectionName}/docs/upsert" \
-H 'content-type: application/json' \
-H 'x-api-key: $LAMBDADB_PROJECT_API_KEY' \
-d '{
"docs": [
{
"id": "33201222",
"url": "https://en.wikipedia.org/wiki/LambdaDB",
"title": "LambdaDB",
"text": "LambdaDB is an AI-native database ... ",
"dense_vector" : [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0],
"sparse_vector" : {"LambdaDB": 0.83, "is": 0.1, "a": 0.1, "AI": 0.7}
},
{
"url": "https://en.wikipedia.org/wiki/Winamp",
"title": "Winamp",
"text": "Winamp is a media player for Windows, macOS and Android ...",
"dense_vector" : [1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 2.0],
"sparse_vector" : {"0": 0.8, "4": 0.1, "11": 0.1, "63": 0.54}
}
]
}'
```
## Response
```json theme={null}
{
"message": "Upsert request is accepted"
}
```
Each document implicitly contains an `id` field in order to uniquely identify a document.
A unique string value is auto-generated by the system if `id` field is not provided in an upsert request.
If you want to overwrite the entire document, you can do so by providing the `id` field in the document.
For a partitioned collection, documents are upserted into the `__default__` partition if the partition field is not provided.
## Upsert limits
| Metric | Limit |
| :--------------------------- | :------------- |
| Max payload size | 6MB |
| Max length for a document ID | 512 characters |
| Max vector dimensions | 4,096 |
| Max document size | 5MB |
When upserting larger amounts of data, it is recommended to use
[bulk-upsert](/guides/documents/bulk-upsert-data) operation.
# Architecture
Source: https://docs.lambdadb.ai/guides/get-started/architecture
Understand LambdaDB's serverless architecture on AWS, including the gateway layer, control and data functions, write buffering, and read caching.
LambdaDB operates as a collection of serverless functions and resources within AWS, completely separating database logic from infrastructure. User requests flow through a regional Gateway, which routes them to either Control or Data functions. The Builder function periodically persists all buffered data to S3 storage.
![LambdaDB serverless architecture diagram]()
### Gateway
**Gateway** verifies the project API key in user requests for targeted projects. If the key is valid, it checks whether the project exceeds its configured rate limit. It then routes the request to either Control or Data functions, based on the type of work needed.
### Control functions
**Control functions** handle project/collection CRUD operations and data management requests such as point-in-time restore and zero-copy fork. They also perform maintenance tasks, such as adjusting the number of virtual shards for each collection to enable parallel query execution based on collection size, triggered by EventBridge Scheduler. They use DynamoDB for storing metadata and conducting distributed coordination among concurrent readers, writers, and background tasks.
### Data functions
**Data functions** perform actual data writes and reads.
#### The write path
When the Writer function receives a request to upsert, update, or delete records in a collection, it records the request details in a log along with a monotonically increasing sequence number. This request log is written into a durable, serverless write buffer (EFS) before returning a response to the client. Later, the Builder function writes the buffered logs to S3 in batches and deletes them once the data is successfully committed. In S3, the data is organized as a tree structure where a root object contains intermediate objects pointing to leaf objects that store the actual data. So the root object basically acts like a commit point that always contains a consistent collection view. This on-storage structure, combined with S3 versioning and lifecycle policies, enables us to implement multi-version concurrency control and advanced features like point-in-time restore efficiently and robustly without reinventing the wheel.
#### The read path
When a query is received, the Router function validates it and then invokes Executor functions based on the number of virtual shards assigned to the collection by a control function. If the client specifies strong consistency, the router also runs the query against buffered logs. Each executor scans its assigned shard data and returns a list of top candidates to the router. The shard data is typically cached in the executor's memory and local storage. If data isn't cached, the executor fetches it from S3 in block units and caches it for future queries. The router then compiles all results, merges and deduplicates them with results from buffered logs if needed, selects the final `top_k` candidates and returns them to the client.
# What is LambdaDB?
Source: https://docs.lambdadb.ai/guides/get-started/overview
Learn about LambdaDB, an AI-native serverless database with built-in vector search, full-text search, hybrid search, and flexible document storage.
LambdaDB is an **AI-native database** designed for building accurate, secure, and scalable AI apps and agents.
Store and search unstructured data at scale without managing any infrastructure.
## 🗃️ Serverless storage & retrieval
Store text, embeddings, and their metadata with flexible schema support across 9 different index types.
k-NN search with cosine, euclidean, dot product, max inner product metrics.
Multi-language analysis (English, Korean, Japanese, standard).
Combines vector + lexical with score normalization (RRF, Min-Max, L2).
Search across multiple vector fields simultaneously.
Efficient high-dimensional sparse vector storage and search.
## 🏢 Enterprise features
### Backup & recovery
* **Continuous backups**: Automatic collection-level backups with 30-day retention by default.
* **Point-in-time recovery (PITR)**: Restore a collection to any specific moment within the retention period.
* **Zero-copy branching**: Instant collection fork without data duplication.
### Operation & management
* **Serverless-native architecture**: Zero infrastructure management required without paying for idle resources.
* **Configurable rate limiting**: Project-level controls for API usage.
* **RESTful API**: Simple HTTP-based interface with comprehensive SDKs.
# Quickstart
Source: https://docs.lambdadb.ai/guides/get-started/quickstart
Get started with LambdaDB in minutes. Create a project, install the Python, TypeScript, or Go SDK, set up your first collection, and run a hybrid search query.
This guide will walk you through getting a **project API key**, installing the SDK, creating your first collection, and running hybrid search queries. You'll learn to set up collections with text, keyword, and dense vector components, then execute both full-text and hybrid searches that combine traditional search with modern vector similarity.
## 🔑 Step 1: Get your API key
You'll use a **project API key** from **[LambdaDB Cloud](https://app.lambdadb.ai)** starting in Step 3.
**LambdaDB Cloud** is in **public preview**.
1. **Sign in to LambdaDB Cloud** — Open [app.lambdadb.ai](https://app.lambdadb.ai), sign up if needed, and sign in.
2. **Create a project** — Accounts without a payment method are on the **Free** plan. Choose an AWS region that fits your latency or data-residency needs.
3. **Copy your project API key** — The API key is shown only once after the project is created. Store it somewhere safe, then use it with the **base URL** and **project name** from the console starting in Step 3.
LambdaDB Cloud uses region-specific API base URLs. Use the **base URL**, **project name**, and **project API key** shown for your project in the LambdaDB Cloud console. Do not assume a global default URL or a fixed project name.
The Free plan includes monthly read, write, storage, and inference usage at no cost. Add a payment method when you need Standard plan usage beyond the Free plan limits. See [Understanding costs](/guides/costs/understanding-costs).
The Cloud console also supports loading data and running queries in the GUI, alongside the SDK examples in this guide.
Keep your API key out of source control and prefer environment variables instead of hardcoding it in scripts.
## 🚀 Step 2: Install the SDK
The LambdaDB SDK provides convenient access to the LambdaDB APIs.
```python Python theme={null}
pip install lambdadb
```
```typescript TypeScript theme={null}
npm install @functional-systems/lambdadb
```
```go Go theme={null}
go get github.com/lambdadb/go-lambdadb
```
For Python, we recommend using a virtual environment to keep your dependencies organized and avoid conflicts between projects.
## 📚 Step 3: Create a collection
A collection is where you'll store your documents and define how they should be indexed for search. LambdaDB supports 9 different index types: text, keyword, long, double, boolean, object, datetime, dense vector, and sparse vector.
Let's create a collection that combines text search with vector similarity:
```python Python theme={null}
from lambdadb import LambdaDB, models
# Initialize the LambdaDB client with the base URL and project name from your LambdaDB Cloud project
with LambdaDB(
project_api_key="your_api_key_here",
base_url="YOUR_BASE_URL",
project_name="YOUR_PROJECT_NAME",
) as client:
collection_name = "your_collection_name_here"
res = client.collections.create(
collection_name=collection_name,
index_configs={
"text": {
"type": "text",
"analyzers": ["english", "korean"],
},
"keyword": {"type": "keyword"},
"vector": {
"type": "vector",
"dimensions": 10,
"similarity": "cosine",
},
},
)
print(res)
```
Python: `LambdaDB` supports context manager usage. `__enter__` returns the client, and `__exit__` calls `client.close()` (closing the SDK-owned HTTP client) and makes the client unusable after the `with` block. If you don't use `with`, call `client.close()` when you're done. If you pass a custom `client=`/`async_client=`, you own closing it.
```typescript TypeScript theme={null}
import { LambdaDBClient } from "@functional-systems/lambdadb";
// Initialize the LambdaDB client with the base URL and project name from your LambdaDB Cloud project
const client = new LambdaDBClient({
projectApiKey: "your_api_key_here",
baseUrl: "YOUR_BASE_URL",
projectName: "YOUR_PROJECT_NAME",
});
const collectionName = "your_collection_name_here";
const res = await client.createCollection({
collectionName,
indexConfigs: {
text: { type: "text", analyzers: ["english", "korean"] },
keyword: { type: "keyword" },
vector: { type: "vector", dimensions: 10, similarity: "cosine" },
},
});
console.log(res);
```
```go Go theme={null}
package main
import (
"context"
"fmt"
"log"
lambdadb "github.com/lambdadb/go-lambdadb"
)
func main() {
ctx := context.Background()
client := lambdadb.New(
lambdadb.WithBaseURL("YOUR_BASE_URL"),
lambdadb.WithProjectName("YOUR_PROJECT_NAME"),
lambdadb.WithAPIKey("your_api_key_here"),
)
collectionName := "your_collection_name_here"
res, err := client.Collections.Create(ctx, lambdadb.CreateCollectionOptions{
CollectionName: collectionName,
IndexConfigs: map[string]interface{}{
"text": map[string]interface{}{"type": "text", "analyzers": []string{"english", "korean"}},
"keyword": map[string]interface{}{"type": "keyword"},
"vector": map[string]interface{}{"type": "vector", "dimensions": 10, "similarity": "cosine"},
},
})
if err != nil {
log.Fatal(err)
}
if res != nil && res.Collection != nil {
c := res.Collection
log.Printf("Collection: project=%s collection=%s numDocs=%d status=%s", c.ProjectName, c.CollectionName, c.NumDocs, c.CollectionStatus)
}
}
```
Response:
```python Python theme={null}
CreateCollectionResponse(
collection=CollectionResponse(
project_name='your-project',
collection_name='quickstart',
index_configs={
'text': IndexConfigsText(
type=,
analyzers=[, ]
),
'keyword': IndexConfigs(type=),
'vector': IndexConfigsVector(
type=,
dimensions=10,
similarity=
),
'id': IndexConfigs(type=)
},
num_docs=0,
collection_status=,
source_project_name=None,
source_collection_name=None,
source_collection_version_id=None
)
)
```
```typescript TypeScript theme={null}
{
collection: {
projectName: 'your-project',
collectionName: 'quickstart',
indexConfigs: { text: [Object], keyword: [Object], vector: [Object], id: [Object] },
numDocs: 0,
collectionStatus: 'CREATING'
}
}
```
```go Go theme={null}
Collection: project=your-project collection=quickstart numDocs=0 status=CREATING
```
**Key configuration details:**
* **Text field**: Supports multilingual search with English and Korean analyzers.
* **Vector field**: 10-dimensional vectors using cosine similarity.
* **Keyword field**: Added to support exact match filtering.
## 📄 Step 4: Add documents
Now let's add some sample documents. Each document contains text for full-text search, keywords for filtering, and vectors for similarity search:
```python Python theme={null}
with LambdaDB(project_api_key="your_api_key_here", base_url="YOUR_BASE_URL", project_name="YOUR_PROJECT_NAME") as client:
coll = client.collection(collection_name)
docs = [
{"id": "doc1", "text": "Serverless computing does not mean no servers are involved. It refers to a cloud computing model where the server management is abstracted away from developers.", "keyword": "serverless", "vector": [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]},
{"id": "doc2", "text": "Instead, it refers to a cloud computing model where developers can build and run applications without having to manage the underlying infrastructure.", "keyword": "cloud", "vector": [0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1]},
{"id": "doc3", "text": "The key aspect is that developers don't need to explicitly provision or manage servers. The cloud provider handles all server management automatically.", "keyword": ["serverless", "infrastructure"], "vector": [0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2]},
]
res = coll.docs.upsert(docs=docs)
print(res)
```
```typescript TypeScript theme={null}
const client = new LambdaDBClient({ projectApiKey: "your_api_key_here", baseUrl: "YOUR_BASE_URL", projectName: "YOUR_PROJECT_NAME" });
const collection = client.collection(collectionName);
const docs = [
{ id: "doc1", text: "Serverless computing does not mean no servers are involved...", keyword: "serverless", vector: [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0] },
{ id: "doc2", text: "Instead, it refers to a cloud computing model where developers can build and run applications...", keyword: "cloud", vector: [0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1] },
{ id: "doc3", text: "The key aspect is that developers don't need to explicitly provision or manage servers...", keyword: ["serverless", "infrastructure"], vector: [0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2] },
];
const res = await collection.docs.upsert({ docs });
console.log(res);
```
```go Go theme={null}
coll := client.Collection(collectionName)
docs := []map[string]interface{}{
{"id": "doc1", "text": "Serverless computing does not mean no servers...", "keyword": "serverless", "vector": []float64{0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0}},
{"id": "doc2", "text": "Instead, it refers to a cloud computing model...", "keyword": "cloud", "vector": []float64{0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1}},
{"id": "doc3", "text": "The key aspect is that developers don't need to explicitly provision...", "keyword": []string{"serverless", "infrastructure"}, "vector": []float64{0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2}},
}
res, err := coll.Docs().Upsert(ctx, lambdadb.UpsertDocsInput{Docs: docs})
if err != nil { log.Fatal(err) }
if res != nil {
log.Printf("message: %s", res.Message)
}
```
Response:
```python Python theme={null}
MessageResponse(message='Upsert request is accepted')
```
```typescript TypeScript theme={null}
{
message: "Upsert request is accepted"
}
```
```go Go theme={null}
message: Upsert request is accepted
```
**Important notes:**
* **Upsert behavior**: Documents with the same ID will be replaced; new IDs create new documents.
* **Auto-generated IDs**: If you don't provide an ID, one will be generated automatically.
* **Bulk operations**: For large-scale document ingestion (5MB+), use the bulk-upsert functionality.
* **Configurable consistency**: LambdaDB is eventually consistent by default, so there can be a slight delay before new or changed documents are visible to queries.
If your application requires strong (read-after-write) consistency, set `consistentRead` (or `consistent_read` in Python) to `true` when you query or fetch data from a collection.
**Check indexing status:** You can view collection stats to verify that your documents have been indexed:
```python Python theme={null}
res = client.collection(collection_name).get() # or client.collections.get(collection_name=collection_name)
print(res)
```
```typescript TypeScript theme={null}
const res = await client.collection(collectionName).get();
console.log(res);
```
```go Go theme={null}
meta, err := client.Collection(collectionName).Get(ctx)
if err != nil { log.Fatal(err) }
_ = meta
```
## 🔍 Step 5: Full-text search
Let's search for documents that match "I hate managing servers" while filtering for documents tagged exactly with "serverless". This demonstrates LambdaDB's powerful query capabilities:
```python Python theme={null}
with LambdaDB(project_api_key="your_api_key_here", base_url="YOUR_BASE_URL", project_name="YOUR_PROJECT_NAME") as client:
coll = client.collection(collection_name)
res = coll.query(
size=10,
query={
"bool": [
{"queryString": {"query": "I hate managing servers", "defaultField": "text"}},
{"queryString": {"query": "keyword:serverless"}, "occur": "must"},
]
},
consistent_read=True,
)
print("🔍 Search Results:")
# Query responses expose items in `res.docs` (each item includes `doc` and `score`).
# Use `res.documents` if you want document bodies only (no scores).
for item in res.docs:
doc_id = str(item.doc.get("id"))
score = f"{item.score:.2f}"
keyword = str(item.doc.get("keyword"))
text = str(item.doc.get("text", ""))[:60]
print(f"{doc_id:<5} | {score:<5} | {keyword:<15} | {text}...")
```
```typescript TypeScript theme={null}
const collection = client.collection(collectionName);
const res = await collection.query({
size: 10,
query: {
bool: [
{ queryString: { query: "I hate managing servers", defaultField: "text" } },
{ queryString: { query: "keyword:serverless" }, occur: "must" },
],
},
consistentRead: true,
});
console.log("🔍 Search Results:");
for (const item of res.docs ?? []) {
const docId = String(item.doc?.id);
const score = item.score?.toFixed(2);
const keyword = String(item.doc?.keyword);
const text = String(item.doc?.text ?? "").slice(0, 60);
console.log(`${docId.padEnd(5)} | ${score} | ${keyword.padEnd(15)} | ${text}...`);
}
```
```go Go theme={null}
coll := client.Collection(collectionName)
res, err := coll.Query(ctx, lambdadb.QueryCollectionInput{
Size: 10,
Query: map[string]interface{}{
"bool": []interface{}{
map[string]interface{}{"queryString": map[string]interface{}{"query": "I hate managing servers", "defaultField": "text"}},
map[string]interface{}{"queryString": map[string]interface{}{"query": "keyword:serverless"}, "occur": "must"},
},
},
ConsistentRead: lambdadb.Ptr(true),
})
if err != nil { log.Fatal(err) }
log.Println("🔍 Search Results:")
for _, r := range res.Docs {
doc := r.Doc
docID := fmt.Sprintf("%v", doc["id"])
score := r.Score
keyword := fmt.Sprintf("%v", doc["keyword"])
text := fmt.Sprintf("%v", doc["text"])
if len(text) > 60 { text = text[:60] }
log.Printf("%-5s | %.2f | %-15s | %s...", docID, score, keyword, text)
}
```
Response:
```python Python theme={null}
🔍 Search Results:
doc3 | 0.83 | ['serverless', 'infrastructure'] | The key aspect is that developers don't need to explicitly provision or manage servers. The cloud provider handles all server management automatically.
doc1 | 0.80 | serverless | Serverless computing does not mean no servers are involved. It refers to a cloud computing model where the server management is abstracted away from developers.
```
```typescript TypeScript theme={null}
🔍 Search Results:
doc3 | 0.83 | ['serverless', 'infrastructure'] | The key aspect is that developers don't need to explicitly provision or manage servers. The cloud provider handles all server management automatically.
doc1 | 0.80 | serverless | Serverless computing does not mean no servers are involved. It refers to a cloud computing model where the server management is abstracted away from developers.
```
```go Go theme={null}
🔍 Search Results:
doc3 | 0.83 | [serverless infrastructure] | The key aspect is that developers don't need to explicitly provision or manage servers. The cloud provider handles all server management automatically.
doc1 | 0.80 | serverless | Serverless computing does not mean no servers are involved. It refers to a cloud computing model where the server management is abstracted away from developers.
```
Why these results? doc3 scored highest because it directly mentions "manage servers", while doc1 matched on "server management" and "serverless computing".
## 🔍 Step 6: Hybrid search
Now let's combine full-text search with vector similarity for more comprehensive results. This is where LambdaDB really shines:
```python Python theme={null}
with LambdaDB(project_api_key="your_api_key_here", base_url="YOUR_BASE_URL", project_name="YOUR_PROJECT_NAME") as client:
coll = client.collection(collection_name)
res = coll.query(
size=10,
query={
"l2": [
{"queryString": {"query": "I hate managing servers", "defaultField": "text"}},
{"knn": {"field": "vector", "k": 5, "queryVector": [0.9, 0.8, 0.7, 0.6, 0.5, 0.4, 0.3, 0.2, 0.1, 1.0]}},
]
},
consistent_read=True,
)
print("🔄 Hybrid Search Results:")
for item in res.docs:
print(f"{item.doc.get('id')} | {item.score:.2f} | {item.doc.get('text', '')[:50]}...")
```
```typescript TypeScript theme={null}
const res = await collection.query({
size: 10,
query: {
l2: [
{ queryString: { query: "I hate managing servers", defaultField: "text" } },
{ knn: { field: "vector", k: 5, queryVector: [0.9, 0.8, 0.7, 0.6, 0.5, 0.4, 0.3, 0.2, 0.1, 1.0] } },
],
},
consistentRead: true,
});
console.log("🔄 Hybrid Search Results:");
for (const item of res.docs ?? []) {
console.log(`${item.doc?.id} | ${item.score?.toFixed(2)} | ${String(item.doc?.text).slice(0, 50)}...`);
}
```
```go Go theme={null}
res, err = coll.Query(ctx, lambdadb.QueryCollectionInput{
Size: 10,
Query: map[string]interface{}{
"l2": []interface{}{
map[string]interface{}{"queryString": map[string]interface{}{"query": "I hate managing servers", "defaultField": "text"}},
map[string]interface{}{"knn": map[string]interface{}{"field": "vector", "k": 5, "queryVector": []float64{0.9, 0.8, 0.7, 0.6, 0.5, 0.4, 0.3, 0.2, 0.1, 1.0}}},
},
},
ConsistentRead: lambdadb.Ptr(true),
})
if err != nil { log.Fatal(err) }
log.Println("🔄 Hybrid Search Results:")
for _, r := range res.Docs {
doc := r.Doc
docID := fmt.Sprintf("%v", doc["id"])
score := r.Score
keyword := fmt.Sprintf("%v", doc["keyword"])
text := fmt.Sprintf("%v", doc["text"])
if len(text) > 50 { text = text[:50] }
log.Printf("%s | %.2f | %s | %s...", docID, score, keyword, text)
}
```
Response:
```python Python theme={null}
🔄 Hybrid Search Results:
doc3 | 0.66 | serverless,infrastructure | The key aspect is that developers don't need to explicitly provision or manage servers. The cloud provider handles all server management automatically.
doc1 | 0.62 | serverless | Serverless computing does not mean no servers are involved. It refers to a cloud computing model where the server management is abstracted away from developers.
doc2 | 0.33 | cloud | Instead, it refers to a cloud computing model where developers can build and run applications without having to manage the underlying infrastructure.
```
```typescript TypeScript theme={null}
🔄 Hybrid Search Results:
doc3 | 0.66 | serverless,infrastructure | The key aspect is that developers don't need to explicitly provision or manage servers. The cloud provider handles all server management automatically.
doc1 | 0.62 | serverless | Serverless computing does not mean no servers are involved. It refers to a cloud computing model where the server management is abstracted away from developers.
doc2 | 0.33 | cloud | Instead, it refers to a cloud computing model where developers can build and run applications without having to manage the underlying infrastructure.
```
```go Go theme={null}
🔄 Hybrid Search Results:
doc3 | 0.66 | serverless,infrastructure | The key aspect is that developers don't need to explicitly provision or manage servers. The cloud provider handles all server management automatically.
doc1 | 0.62 | serverless | Serverless computing does not mean no servers are involved. It refers to a cloud computing model where the server management is abstracted away from developers.
doc2 | 0.33 | cloud | Instead, it refers to a cloud computing model where developers can build and run applications without having to manage the underlying infrastructure.
```
**Score normalization options:**
* `rrf` (Reciprocal Rank Fusion): Great for combining rankings from different search methods
* `l2` (L2 Normalization): Normalizes scores using L2 norm
* `mm` (Min-Max Normalization): Simple linear scaling to 0-1 range
## 🧹 Step 7: Clean up
When you're finished experimenting, clean up your resources:
```python Python theme={null}
client.collections.delete(collection_name=collection_name)
```
```typescript TypeScript theme={null}
await client.collection(collectionName).delete();
```
```go Go theme={null}
err := client.Collection(collectionName).Delete(ctx)
if err != nil { log.Fatal(err) }
```
## 🚀 Next steps
* **Advanced Queries**: Explore complex patterns in our [Query Guide](/guides/search/search-overview)
* **Bulk Operations**: Learn about large-scale data ingestion in our [Bulk Operations Guide](/guides/documents/bulk-upsert-data)
* **API Reference**: Comprehensive documentation at our [API Reference](/reference/api/introduction)
## 🤝 Support
Need help? Visit our [Community Slack](https://join.slack.com/t/lambdadbcommunity/shared_invite/zt-3sg7565zm-sCTW3odRkEQt~auWUVVsTw) for support and discussions.
# Roadmap & Changelog
Source: https://docs.lambdadb.ai/guides/get-started/roadmap-changelog
Track the LambdaDB product roadmap and changelog, including upcoming features like integrated embeddings, partition management, and recent releases.
Last updated: May 21, 2026
## Up Next
* Query and indexing performance, always
* Explicit partition management: add and delete collection partitions beyond hash-based partitioning
* Flexible continuous backup retention: set custom point-in-time recovery windows and update them after collection creation
* MCP support for agentic search and retrieval
## Changelog
### May 2026
* Completed integrated embeddings for managed vector fields. You can now send source text, let LambdaDB generate and store embeddings, and query managed embedding fields with `knn.queryText`. See [Managed embeddings](/guides/collections/managed-embeddings).
* Added inference usage to [costs and Free plan quotas](/guides/costs/understanding-costs). Managed model usage is measured in LIU.
### April 2026
* Added the initial [Free plan](/guides/costs/understanding-costs).
### March 2026
* Launched [LambdaDB Cloud](https://app.lambdadb.ai/auth/signup) public preview across all global AWS regions.
* Released the [Python, TypeScript, and Go SDKs](/reference/sdk/introduction), plus [langchain-lambdadb](https://pypi.org/project/langchain-lambdadb/) and [langchain-lambdadb-js](https://www.npmjs.com/package/@functional-systems/langchain-lambdadb).
# Use with MCP
Source: https://docs.lambdadb.ai/guides/get-started/use-with-mcp
Connect LambdaDB to AI assistants via the Model Context Protocol. Configure MCP clients like Claude Desktop to search and manage collections.
LambdaDB can be used from any **Model Context Protocol (MCP)** client through the **LambdaDB MCP server**. This is useful when you want an AI assistant to inspect collections, list documents, fetch records, or run search queries against a LambdaDB project.
This guide shows a local setup flow using a **project API key**, **base URL**, and **project name** from LambdaDB Cloud.
## Before you start
You will need:
* A LambdaDB **project API key**
* Your project's **base URL**
* Your **project name**
* A local copy of the **LambdaDB MCP server**
* An MCP client such as **Claude Desktop**
The LambdaDB MCP server is designed to connect to a single LambdaDB project per process. It uses a **project-scoped API key** and is **read-only by default** unless write tools are explicitly enabled.
## Step 1: Get your LambdaDB connection details
From **[LambdaDB Cloud](https://app.lambdadb.ai)**, collect the following values for the project you want to expose to your MCP client:
* **Base URL**
* **Project name**
* **Project API key**
Store your project API key securely. Do not commit it to source control or share it in screenshots, prompts, or logs.
## Step 2: Build the LambdaDB MCP server
Clone the LambdaDB MCP server locally, then build it:
```bash theme={null}
git clone https://github.com/lambdadb/lambdadb-mcp
cd lambdadb-mcp
npm install
npm run build
```
If you are using the repository's local environment workflow, you can also create a local `.env` file:
```bash theme={null}
cp .env.example .env
```
Then set:
```bash theme={null}
LAMBDADB_BASE_URL=YOUR_BASE_URL
LAMBDADB_PROJECT_NAME=YOUR_PROJECT_NAME
LAMBDADB_PROJECT_API_KEY=YOUR_PROJECT_API_KEY
LAMBDADB_MCP_ENABLE_WRITE_TOOLS=false
```
Keep `LAMBDADB_MCP_ENABLE_WRITE_TOOLS=false` until you have validated the read-only tools in your client.
## Step 3: Add the server to your MCP client
For **Claude Desktop**, add an MCP server entry that points to the built `dist/index.js` file.
```json theme={null}
{
"mcpServers": {
"lambdadb": {
"command": "node",
"args": ["/absolute/path/to/lambdadb-mcp/dist/index.js"],
"env": {
"LAMBDADB_BASE_URL": "YOUR_BASE_URL",
"LAMBDADB_PROJECT_NAME": "YOUR_PROJECT_NAME",
"LAMBDADB_PROJECT_API_KEY": "YOUR_PROJECT_API_KEY",
"LAMBDADB_MCP_ENABLE_WRITE_TOOLS": "false"
}
}
}
}
```
After saving the configuration, restart your MCP client if needed.
## Step 4: Verify the connection
Once the MCP server is connected, start with the read-only tools:
* `lambdadb_list_collections`
* `lambdadb_get_collection`
* `lambdadb_query_collection`
* `lambdadb_list_docs`
* `lambdadb_fetch_docs`
Recommended first checks:
1. Call `lambdadb_list_collections` to confirm the project connection.
2. Call `lambdadb_get_collection` for a known collection.
3. Run `lambdadb_query_collection` with a small query against that collection.
## Optional: Enable write tools
The LambdaDB MCP server can also expose write operations such as:
* `lambdadb_create_collection`
* `lambdadb_upsert_docs`
* `lambdadb_delete_docs`
To enable them, change:
```bash theme={null}
LAMBDADB_MCP_ENABLE_WRITE_TOOLS=true
```
Enable write tools only in environments where document creation, updates, and deletion are intended. Start with a non-production project when testing write access from an MCP client.
## Troubleshooting
If the server starts but tools fail:
* Verify that **base URL**, **project name**, and **project API key** all belong to the same LambdaDB project.
* Confirm that the project API key is still valid.
* Check the MCP client logs for startup or tool-call errors.
* Run the server locally first to make sure the process starts with the expected environment variables.
If tools are missing:
* Confirm that the MCP client loaded the correct `dist/index.js` path.
* Restart the MCP client after editing its configuration.
* If you expect write tools, make sure `LAMBDADB_MCP_ENABLE_WRITE_TOOLS=true` is set.
## Next steps
* Learn how to create collections in [Create a collection](/guides/collections/create-a-collection)
* Learn how to list documents in [List data](/guides/documents/list-data)
* Learn how to search collections in [Search overview](/guides/search/search-overview)
# Migrate from Elasticsearch
Source: https://docs.lambdadb.ai/guides/migrations/elasticsearch/migrate-from-elasticsearch
Use the LambdaDB Migration CLI to move Elasticsearch index mappings, documents, and dense vectors to LambdaDB.
Use the LambdaDB Migration CLI to migrate an Elasticsearch index into LambdaDB. The CLI inventories the Elasticsearch mapping, generates an editable LambdaDB mapping, creates the target LambdaDB collection when needed, reads documents with Elasticsearch point-in-time pagination, writes LambdaDB documents, saves local checkpoints, and validates migrated records before cutover.
Elasticsearch support is available in LambdaDB Migration CLI `v0.1.5` and later. If your installed CLI does not show `lambdadb-migration elasticsearch --help`, install a newer release before following this guide.
This guide assumes you are migrating a concrete Elasticsearch index whose source documents can be returned from `_source`. If you use aliases, data streams, or wildcard patterns that resolve to multiple backing indexes, run one migration per concrete index or prepare a custom consolidation plan.
Elasticsearch stores records as JSON documents inside indexes. Search behavior is controlled by mappings, analyzers, dense vector fields, query DSL, ingest pipelines, aliases, and cluster-level settings. LambdaDB stores each migrated record as a document and indexes only the fields declared in the collection's `indexConfigs`.
## What the CLI supports
| Elasticsearch source | LambdaDB target | Migration behavior |
| :---------------------------------------- | :----------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Index | Collection | Migrate one Elasticsearch index into one LambdaDB collection. |
| `_id` | Document `id` | Copy Elasticsearch `_id` as a LambdaDB string document ID. |
| `_source` fields | Document fields | Copy fields returned in `_source`. Fields removed from `_source`, or indexes with `_source` disabled, cannot be reconstructed by the default migration path. Nested objects are flattened into dot-path payload fields, then normalized for LambdaDB field names through the generated mapping. |
| `text` field | `text` index | Generate a LambdaDB text index for searchable full-text fields. Review analyzer choices before cutover because Elasticsearch analyzers are not copied automatically. |
| `keyword`, `constant_keyword`, `wildcard` | `keyword` index | Generate LambdaDB keyword indexes for exact filters, IDs, categories, tags, and sort keys. |
| Numeric fields | `long` or `double` index | Map integer-like fields to `long` and floating-point fields to `double`. |
| `date`, `date_nanos` | `datetime` index | Generate a LambdaDB datetime index. Review source values before writing because the CLI does not rewrite date formats. |
| `boolean` field | `boolean` index | Copy JSON booleans directly. |
| `dense_vector` field | `vector` index | Generate an unmanaged LambdaDB vector field with mapped dimensions and similarity. Dense vectors are fetched explicitly with Elasticsearch search `fields`. |
| `nested` field | `object` index | Store nested values as JSON payload fields and warn that nested query semantics require application-query review. |
## What still needs review
Review these features before cutover because the CLI migrates data and generated LambdaDB index configs, not Elasticsearch runtime behavior:
* Elasticsearch Query DSL, aggregations, scripts, runtime fields, scoring scripts, and rescoring logic
* custom analyzers, token filters, synonyms, normalizers, and language-specific index settings
* nested query semantics, parent-child relationships, join fields, and field collapsing
* index templates, aliases, data streams, ILM policies, and ingest pipelines
* `semantic_text`, ELSER, model inference pipelines, and other Elasticsearch-managed semantic features
* `_source` exclusions, disabled `_source`, synthetic `_source` differences, and stored-field-only designs
* application code that expects Elasticsearch response shapes, shard metadata, highlights, or aggregations
For search applications, first decide which production queries must move to LambdaDB `queryString`, `knn`, `sparseVector`, `bool`, or hybrid queries. Then review the generated LambdaDB mapping around those queries instead of copying every Elasticsearch mapping field mechanically.
## Step 1: Set credentials
LambdaDB Cloud uses region-specific API base URLs. Use the **base URL**, **project name**, and **project API key** shown for your project in the LambdaDB Cloud console. Do not assume a global default URL or a fixed project name.
Set your Elasticsearch API key and LambdaDB connection values:
```bash theme={null}
export ELASTIC_API_KEY="YOUR_ELASTICSEARCH_API_KEY"
export LAMBDADB_BASE_URL="YOUR_REGION_BASE_URL"
export LAMBDADB_PROJECT_NAME="YOUR_PROJECT_NAME"
export LAMBDADB_PROJECT_API_KEY="YOUR_PROJECT_API_KEY"
```
You can also pass Elasticsearch basic auth credentials with `--elasticsearch.username` and `--elasticsearch.password`.
## Step 2: Generate inventory and mapping
Run the inventory command against the Elasticsearch endpoint and index:
```bash theme={null}
lambdadb-migration inventory elasticsearch \
--elasticsearch.url https://your-elasticsearch-endpoint \
--elasticsearch.index articles \
--output elasticsearch-inventory.yaml
```
The output includes the source inventory and an editable LambdaDB mapping:
```yaml theme={null}
inventory:
sourceKind: elasticsearch
collectionName: articles
recordCount: 250000
vectors:
embedding:
name: embedding
dimensions: 768
similarity: cosine
payloadIndexes:
title:
name: title
type: text
metadata.source:
name: metadata.source
type: keyword
mapping:
target:
collection: articles
createCollection: true
vectors:
embedding:
targetField: embedding
dimensions: 768
similarity: cosine
payload:
mode: flatten
rename:
metadata.source: metadata_source
indexConfigs:
title:
type: text
metadata_source:
type: keyword
ids:
targetField: id
```
Review warnings in the inventory output. Common warnings include unsupported mapping types, Elasticsearch multi-fields that are not copied as separate source fields, `nested` fields that need query rewrite review, and PIT checkpoint expiry.
Use a concrete index name for inventory and migration. If an alias or wildcard resolves to multiple indexes, the CLI can only generate one LambdaDB mapping from the mapping response, while the source count and PIT read may cover more data than that single generated mapping represents.
## Step 3: Review the mapping
Review the generated mapping before migration:
* confirm the target LambdaDB collection name
* remove payload index configs for fields you only need to store
* check generated renames for dotted fields such as `metadata.source`
* confirm `dense_vector` dimensions and similarity
* add LambdaDB `text` analyzers such as `english`, `korean`, or `japanese` when your Elasticsearch workload depends on language-specific analysis
* confirm `date` and `date_nanos` values in `_source` are compatible with LambdaDB `datetime` fields, or edit the mapping before creating the collection
* add explicit field decisions for Elasticsearch multi-fields such as `title.keyword` if your application depends on exact-match behavior
The CLI maps Elasticsearch `l2_norm` vector similarity to LambdaDB `euclidean`. Elasticsearch `cosine`, `dot_product`, and `max_inner_product` map directly.
If you want to fetch vector fields explicitly instead of relying on mapping discovery, pass a comma-separated list:
```bash theme={null}
--elasticsearch.vector-fields embedding,title_embedding
```
## Step 4: Run a dry run
Run a dry run to validate the source inventory and mapping without writing documents:
```bash theme={null}
lambdadb-migration elasticsearch \
--elasticsearch.url https://your-elasticsearch-endpoint \
--elasticsearch.index articles \
--lambdadb.base-url "$LAMBDADB_BASE_URL" \
--lambdadb.project-name "$LAMBDADB_PROJECT_NAME" \
--lambdadb.api-key "$LAMBDADB_PROJECT_API_KEY" \
--lambdadb.collection articles \
--mapping-file elasticsearch-inventory.yaml \
--migration.dry-run
```
## Step 5: Run the migration
Run the migration with validation enabled:
```bash theme={null}
lambdadb-migration elasticsearch \
--elasticsearch.url https://your-elasticsearch-endpoint \
--elasticsearch.index articles \
--lambdadb.base-url "$LAMBDADB_BASE_URL" \
--lambdadb.project-name "$LAMBDADB_PROJECT_NAME" \
--lambdadb.api-key "$LAMBDADB_PROJECT_API_KEY" \
--lambdadb.collection articles \
--mapping-file elasticsearch-inventory.yaml \
--migration.write-mode bulk \
--migration.validate \
--migration.validation-report validation-report.json
```
The Elasticsearch connector reads documents with a point in time (PIT), pages with `search_after`, and sorts by `_shard_doc`. It stores the latest PIT ID and `search_after` values in the local checkpoint.
For large indexes or slow write targets, increase the PIT keep-alive window:
```bash theme={null}
--elasticsearch.pit-keep-alive 15m
```
Elasticsearch PIT IDs can expire. If a resumed migration fails because the saved PIT is no longer valid, rerun with `--migration.restart`.
## Step 6: Validate results
Validation compares the accepted record count against the Elasticsearch inventory count, fetches sampled migrated documents from LambdaDB with strongly consistent reads, and compares sampled fields.
Use a validation report for review:
```bash theme={null}
--migration.validate \
--migration.validation-report validation-report.json
```
`--migration.query-overlap` currently supports Qdrant and Pinecone sources, not Elasticsearch. For Elasticsearch migrations, use count validation, sampled document validation, and manual review of representative LambdaDB search queries.
Example LambdaDB lexical query:
```json JSON theme={null}
{
"queryString": {
"query": "serverless database",
"defaultField": "body"
}
}
```
Example LambdaDB hybrid query:
```json JSON theme={null}
{
"rrf": [
{
"queryString": {
"query": "serverless database",
"defaultField": "body"
}
},
{
"knn": {
"field": "embedding",
"queryVector": [0.1, 0.2, 0.3],
"k": 10
}
}
]
}
```
## Step 7: Cut over safely
Run the LambdaDB path in parallel before replacing production Elasticsearch traffic:
1. Backfill historical documents.
2. Replay writes that happened during the backfill window.
3. Dual-write new updates to Elasticsearch and LambdaDB for a short verification period.
4. Compare representative query results and latency.
5. Switch read traffic to LambdaDB.
6. Keep Elasticsearch available until rollback is no longer needed.
## Elasticsearch references
* [Paginate search results](https://www.elastic.co/docs/reference/elasticsearch/rest-apis/paginate-search-results)
* [Dense vector field type](https://www.elastic.co/docs/reference/elasticsearch/mapping-reference/dense-vector)
* [Text field type](https://www.elastic.co/docs/reference/elasticsearch/mapping-reference/text)
* [Keyword type family](https://www.elastic.co/docs/reference/elasticsearch/mapping-reference/keyword)
## Related docs
Learn the shared migration workflow, validation behavior, and checkpoint behavior.
Define the LambdaDB index configuration for migrated fields.
Rewrite Elasticsearch lexical queries to LambdaDB query string syntax.
Combine lexical and vector search after migration.
# Migration CLI
Source: https://docs.lambdadb.ai/guides/migrations/overview
Use the LambdaDB Migration CLI to move vector search workloads from existing systems into LambdaDB.
The recommended path for moving existing vector search workloads to LambdaDB is the LambdaDB Migration CLI.
The CLI inventories the source system, generates an editable LambdaDB mapping, creates the target collection when needed, streams records into LambdaDB, saves local checkpoints, and can validate migrated documents before cutover.
## Supported sources
| Source | Status | Notes |
| :------------------ | :------------------------ | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Qdrant | Supported in CLI `v0.1.5` | Dense vectors, named dense vectors, sparse vectors, payload indexes, local checkpoints, validation reports, and dense/sparse-vector query overlap checks. |
| Pinecone Serverless | Supported in CLI `v0.1.5` | Dense indexes, sparse indexes, namespaces, ID-prefix scoped migrations, metadata payloads, local checkpoints, validation reports, and dense/sparse-vector query overlap checks. |
| Elasticsearch | Supported in CLI `v0.1.5` | Index mappings, scalar/text fields, dense vectors, PIT/search\_after reads, local checkpoints, and validation reports. Query-overlap validation is not implemented for Elasticsearch yet. |
| OpenSearch | Planned | Use the same CLI workflow once OpenSearch support is available. |
| Chroma | Planned | Use the same CLI workflow once Chroma support is available. |
| Weaviate | Planned | Use the same CLI workflow once Weaviate support is available. |
## Migration workflow
1. Install the CLI.
2. Generate an inventory and editable mapping from the source.
3. Review the generated LambdaDB collection and field mapping.
4. Run a dry run.
5. Run the migration with validation enabled.
6. Review the validation report and representative search results.
7. Switch production traffic after result quality, latency, and application query rewrites are verified.
## Install the CLI
Install the latest release:
```bash theme={null}
curl -fsSL https://raw.githubusercontent.com/lambdadb/lambdadb-migration/main/install.sh | sh
```
Install a specific version:
```bash theme={null}
curl -fsSLO https://raw.githubusercontent.com/lambdadb/lambdadb-migration/main/install.sh
sh install.sh --version v0.1.5 --install-dir "$HOME/.local/bin"
```
Make sure the install directory is on your `PATH`, then check the CLI:
```bash theme={null}
lambdadb-migration --help
```
## Common CLI behavior
LambdaDB Cloud uses region-specific API base URLs. Use the **base URL**, **project name**, and **project API key** shown for your project in the LambdaDB Cloud console. Do not assume a global default URL or a fixed project name.
Set these values before running migration commands:
```bash theme={null}
export LAMBDADB_BASE_URL="YOUR_REGION_BASE_URL"
export LAMBDADB_PROJECT_NAME="YOUR_PROJECT_NAME"
export LAMBDADB_PROJECT_API_KEY="YOUR_PROJECT_API_KEY"
```
Generated mappings set `target.createCollection: true` by default. With that setting, the migration creates the LambdaDB collection if it does not already exist, then waits until the collection is ready before writing documents.
Use `--migration.create-collection=false` when the LambdaDB collection already exists and the migration should fail instead of creating it.
The CLI stores local checkpoints under `.lambdadb-migration/checkpoints` by default. If a migration is interrupted, rerun the same command to resume from the last saved checkpoint. Use `--migration.restart` to start from the beginning.
## Validation
Use `--migration.validate` for post-migration checks. Validation compares the accepted record count against the source inventory, fetches a sample of migrated documents from LambdaDB with strongly consistent reads, and compares sampled fields.
Use `--migration.validation-report` to write a JSON report:
```bash theme={null}
--migration.validate \
--migration.validation-report validation-report.json
```
For Qdrant and Pinecone vector migrations, `--migration.query-overlap` compares source and LambdaDB nearest-neighbor results for validation samples. By default, it reports overlap without failing the migration. Set `--migration.query-overlap-min-ratio` above `0` to require a minimum average overlap. Query-overlap validation is not implemented for Elasticsearch yet.
## Next steps
Move Qdrant collections, points, vectors, sparse vectors, and payload indexes to LambdaDB.
Move Pinecone Serverless indexes, namespaces, vectors, and metadata to LambdaDB.
Move Elasticsearch index mappings, documents, and dense vectors to LambdaDB.
Learn how LambdaDB collection index configurations map to migrated data.
Understand LambdaDB bulk loading behavior for unmanaged-vector collections.
Rewrite dense, sparse, and lexical hybrid search after migration.
# Migrate from Pinecone
Source: https://docs.lambdadb.ai/guides/migrations/pinecone/migrate-from-pinecone
Use the LambdaDB Migration CLI to move Pinecone Serverless indexes, namespaces, dense vectors, sparse vectors, and metadata to LambdaDB.
Use the LambdaDB Migration CLI to migrate Pinecone Serverless indexes into LambdaDB. The CLI inventories your Pinecone index, generates an editable LambdaDB mapping, creates the target LambdaDB collection when needed, lists and fetches Pinecone records, writes LambdaDB documents, saves local checkpoints, and validates migrated records before cutover.
Pinecone vector-API records contain an `id`, dense `values` and/or `sparse_values`, and optional `metadata`. Records live inside namespaces. LambdaDB stores each migrated record as a **document**: the Pinecone ID becomes the LambdaDB document `id`, metadata becomes document fields, and vector values become LambdaDB `vector` or `sparseVector` fields.
## What the CLI supports
| Pinecone data | LambdaDB target | Migration behavior |
| :--------------------------- | :-------------------------- | :-------------------------------------------------------------------------------------------------------------------------------- |
| Serverless index | Collection | Migrate one Pinecone index or namespace into one LambdaDB collection. |
| Namespace | Migration scope | Pass `--pinecone.namespace` to migrate a single namespace. The namespace name is not added to documents automatically. |
| Record ID | Document `id` | Pinecone record IDs are copied as LambdaDB string document IDs. |
| Dense vector values | `vector` field | The generated mapping uses `dense` as the target field for standard dense indexes. |
| Sparse vector values | `sparseVector` field | Pinecone `indices` and `values` arrays are converted to a LambdaDB sparse object. |
| Metadata | Document fields | Metadata fields are flattened by default. Dotted field names are normalized, for example `metadata.url` to `metadata_url`. |
| Metadata indexing | `payload.indexConfigs` | Pinecone metadata index settings are not currently introspected. Add LambdaDB index configs for fields you need to query or sort. |
| Integrated embedding indexes | Stored vectors and metadata | The CLI migrates stored vector values and metadata, not Pinecone hosted model configuration. |
The Pinecone connector uses Pinecone's vector listing API, which is available for Serverless indexes. Pod-based indexes and workloads that cannot list vector IDs should be moved to Pinecone Serverless first or migrated with a custom export path.
Pinecone's newer document-schema indexes can mix `dense_vector`, `sparse_vector`, full-text `string`, and metadata fields. The current LambdaDB Migration CLI path is designed around Pinecone Serverless vector records that can be listed and fetched. Review document-schema and full-text workloads before using the default migration path.
Pinecone's current API reference is versioned as `2026-04`. If you call Pinecone REST APIs directly during a custom migration, set the `X-Pinecone-Api-Version` header explicitly. The LambdaDB Migration CLI uses Pinecone's official SDK and does not require you to pass this header.
## Step 1: Install the CLI
Install the latest release:
```bash theme={null}
curl -fsSL https://raw.githubusercontent.com/lambdadb/lambdadb-migration/main/install.sh | sh
```
Install a specific version:
```bash theme={null}
curl -fsSLO https://raw.githubusercontent.com/lambdadb/lambdadb-migration/main/install.sh
sh install.sh --version v0.1.5 --install-dir "$HOME/.local/bin"
```
Check the Pinecone command:
```bash theme={null}
lambdadb-migration pinecone --help
```
## Step 2: Set credentials
LambdaDB Cloud uses region-specific API base URLs. Use the **base URL**, **project name**, and **project API key** shown for your project in the LambdaDB Cloud console. Do not assume a global default URL or a fixed project name.
Set your Pinecone API key and LambdaDB connection values:
```bash theme={null}
export PINECONE_API_KEY="YOUR_PINECONE_API_KEY"
export LAMBDADB_BASE_URL="YOUR_REGION_BASE_URL"
export LAMBDADB_PROJECT_NAME="YOUR_PROJECT_NAME"
export LAMBDADB_PROJECT_API_KEY="YOUR_PROJECT_API_KEY"
```
If you use a non-default Pinecone control-plane host, pass it with `--pinecone.host`.
## Step 3: Generate inventory and mapping
Run the inventory command against a Pinecone Serverless index:
```bash theme={null}
lambdadb-migration inventory pinecone \
--pinecone.index articles \
--pinecone.namespace production \
--output pinecone-inventory.yaml
```
Use `--pinecone.list-prefix` when you only want to migrate records whose IDs start with a specific prefix:
```bash theme={null}
lambdadb-migration inventory pinecone \
--pinecone.index articles \
--pinecone.namespace production \
--pinecone.list-prefix "tenant-a#" \
--output pinecone-inventory.yaml
```
A dense-vector inventory produces an editable mapping like this:
```yaml theme={null}
mapping:
target:
collection: articles
createCollection: true
vectors:
"":
targetField: dense
dimensions: 1536
similarity: cosine
sparseVectors: {}
payload:
mode: flatten
rename: {}
indexConfigs: {}
ids:
targetField: id
```
Review the generated mapping before running the migration. In particular:
* Add `payload.indexConfigs` for metadata fields that must be searchable or sortable in LambdaDB.
* Check dense vector dimensions and similarity.
* Check normalized dotted metadata fields.
* Decide whether one Pinecone namespace should become one LambdaDB collection, or whether you should run multiple scoped migrations.
For example, add indexes for metadata fields used by filters:
```yaml theme={null}
mapping:
target:
collection: articles
createCollection: true
vectors:
"":
targetField: dense
dimensions: 1536
similarity: cosine
sparseVectors: {}
payload:
mode: flatten
rename: {}
indexConfigs:
tenant_id:
type: keyword
title:
type: text
analyzers: ["english"]
created_at:
type: datetime
metadata_url:
type: keyword
ids:
targetField: id
```
For a Pinecone sparse index, the generated mapping includes a sparse vector field:
```yaml theme={null}
mapping:
target:
collection: article-keywords
createCollection: true
vectors: {}
sparseVectors:
sparse:
targetField: sparse
payload:
mode: flatten
rename: {}
indexConfigs: {}
ids:
targetField: id
```
Generated mappings set `target.createCollection: true` by default. With that setting, the migration creates the LambdaDB collection if it is missing and waits until it is ready before writing documents.
## Step 4: Run a dry run
Use a dry run to validate the mapping and inspect the planned migration without writing documents:
```bash theme={null}
lambdadb-migration pinecone \
--pinecone.index articles \
--pinecone.namespace production \
--lambdadb.base-url "$LAMBDADB_BASE_URL" \
--lambdadb.project-name "$LAMBDADB_PROJECT_NAME" \
--lambdadb.api-key "$LAMBDADB_PROJECT_API_KEY" \
--lambdadb.collection articles \
--mapping-file pinecone-inventory.yaml \
--migration.dry-run
```
## Step 5: Run the migration
Run the migration with validation enabled:
```bash theme={null}
lambdadb-migration pinecone \
--pinecone.index articles \
--pinecone.namespace production \
--lambdadb.base-url "$LAMBDADB_BASE_URL" \
--lambdadb.project-name "$LAMBDADB_PROJECT_NAME" \
--lambdadb.api-key "$LAMBDADB_PROJECT_API_KEY" \
--lambdadb.collection articles \
--mapping-file pinecone-inventory.yaml \
--migration.write-mode bulk \
--migration.validate \
--migration.validation-report validation-report.json
```
Migration progress is written to stderr with accepted count, percent, batch size, rate, and elapsed time.
The CLI stores checkpoints under `.lambdadb-migration/checkpoints` by default. If the command is interrupted, rerun the same command to resume. Use `--migration.restart` to ignore an existing checkpoint and start from the beginning.
Use `--migration.create-collection=false` when the target LambdaDB collection already exists and the migration should not create it.
## Step 6: Review validation
`--migration.validate` checks the accepted record count, fetches a sample of migrated documents from LambdaDB using strongly consistent reads, and compares sampled fields.
`--migration.validation-report` writes the validation result as JSON. The report includes source count, accepted records, LambdaDB `numDocs`, sampled IDs, compared sample count, query overlap results, and validation errors.
For dense or sparse-vector migrations, add query overlap checks:
```bash theme={null}
lambdadb-migration pinecone \
--pinecone.index articles \
--pinecone.namespace production \
--lambdadb.base-url "$LAMBDADB_BASE_URL" \
--lambdadb.project-name "$LAMBDADB_PROJECT_NAME" \
--lambdadb.api-key "$LAMBDADB_PROJECT_API_KEY" \
--lambdadb.collection articles \
--mapping-file pinecone-inventory.yaml \
--migration.validate \
--migration.validation-report validation-report.json \
--migration.query-overlap
```
By default, `--migration.query-overlap` reports vector overlap without failing the migration. Set `--migration.query-overlap-min-ratio` above `0` to require a minimum average overlap.
## Mapping details
Use this metric mapping when creating LambdaDB vector fields:
| Pinecone metric | LambdaDB similarity |
| :-------------- | :------------------ |
| `cosine` | `cosine` |
| `euclidean` | `euclidean` |
| `dotproduct` | `dot_product` |
For a Pinecone dense record:
```json Pinecone record theme={null}
{
"id": "doc-1",
"values": [0.12, 0.34, 0.56],
"metadata": {
"tenant_id": "acme",
"title": "Refund policy",
"body": "Refunds are available within 7 days.",
"created_at": "2026-05-01T10:00:00Z"
}
}
```
the CLI writes a LambdaDB document like this:
```json LambdaDB document theme={null}
{
"id": "doc-1",
"tenant_id": "acme",
"title": "Refund policy",
"body": "Refunds are available within 7 days.",
"created_at": "2026-05-01T10:00:00Z",
"dense": [0.12, 0.34, 0.56]
}
```
For a Pinecone sparse record:
```json Pinecone sparse values theme={null}
{
"id": "doc-1",
"sparse_values": {
"indices": [3, 9],
"values": [0.7, 0.2]
},
"metadata": {
"title": "Refund policy"
}
}
```
the CLI converts sparse values to an object whose keys are index strings:
```json LambdaDB document theme={null}
{
"id": "doc-1",
"title": "Refund policy",
"sparse": {
"3": 0.7,
"9": 0.2
}
}
```
## Rewrite vector search
A Pinecone dense-vector query:
```python Python theme={null}
index.query(
namespace="production",
vector=query_vector,
top_k=10,
filter={"tenant_id": {"$eq": "acme"}},
include_metadata=True,
include_values=False,
)
```
becomes a LambdaDB `knn` query:
```python Python theme={null}
results = coll.query(
query={
"knn": {
"field": "dense",
"queryVector": query_vector,
"k": 10,
"filter": {
"queryString": {
"query": '"acme"',
"defaultField": "tenant_id",
}
},
}
},
size=10,
)
```
If you use LambdaDB managed embeddings instead of migrated vector values, send query text:
```python Python theme={null}
results = coll.query(
query={
"knn": {
"field": "bodyEmbedding",
"queryText": "refund policy",
"k": 10,
}
},
size=10,
)
```
## Rewrite sparse and hybrid search
A Pinecone sparse-vector query:
```python Python theme={null}
index.query(
namespace="production",
sparse_vector={"indices": [3, 9], "values": [0.7, 0.2]},
top_k=10,
include_metadata=True,
include_values=False,
)
```
becomes a LambdaDB sparse vector query:
```python Python theme={null}
results = coll.query(
query={
"sparseVector": {
"field": "sparse",
"queryVector": {
"3": 0.7,
"9": 0.2,
},
}
},
size=10,
)
```
Pinecone vector-API hybrid search can use one index with both dense and sparse values, or separate dense and sparse indexes whose results are merged client-side. In LambdaDB, use a collection with both dense and sparse fields, then query those fields together:
```json LambdaDB dense + sparse hybrid query theme={null}
{
"mm": [
{
"knn": {
"field": "dense",
"queryVector": [0.01, 0.45, 0.67],
"k": 20
},
"boost": 0.7
},
{
"sparseVector": {
"field": "sparse",
"queryVector": {
"3": 0.7,
"9": 0.2
}
},
"boost": 0.3
}
]
}
```
Use `rrf` when you want rank fusion without explicit boosts, or `mm`/`l2` when you want weighted score fusion.
## Rewrite filters
Pinecone metadata filters use operators such as `$eq`, `$gte`, `$in`, `$and`, and `$or`. In LambdaDB, map simple exact and range filters to `queryString`, and use `bool` when you need multiple clauses.
```json Pinecone metadata filter theme={null}
{
"$and": [
{ "tenant_id": { "$eq": "acme" } },
{ "year": { "$gte": 2024 } },
{ "status": { "$ne": "deleted" } }
]
}
```
```json LambdaDB bool query theme={null}
{
"bool": [
{
"queryString": {
"query": "tenant_id:acme"
},
"occur": "filter"
},
{
"queryString": {
"query": "year:[2024 TO *]"
},
"occur": "filter"
},
{
"queryString": {
"query": "status:deleted"
},
"occur": "must_not"
}
]
}
```
Use field types intentionally:
| Pinecone metadata use | LambdaDB index type |
| :---------------------------------------- | :------------------ |
| Exact string match, tags, IDs, tenant IDs | `keyword` |
| Natural-language matching | `text` |
| Integer range or sorting | `long` |
| Floating-point range or sorting | `double` |
| Date/time range or sorting | `datetime` |
| Boolean flags | `boolean` |
| Nested JSON kept as a searchable object | `object` |
## Common gotchas
* **Serverless only**: Pinecone's `list` endpoint is supported only for Serverless indexes. The CLI depends on vector listing, then fetches listed records by ID.
* **Namespaces**: Run one migration per namespace. If the namespace itself matters in LambdaDB, use separate target collections or add namespace metadata in Pinecone before migration.
* **Metadata indexes**: Pinecone metadata index settings are not introspected. Add LambdaDB `payload.indexConfigs` manually for fields used in filters or sorting.
* **Field names**: LambdaDB field names cannot contain dots. Dotted Pinecone metadata keys are normalized during migration, such as `metadata.url` to `metadata_url`.
* **Integrated embeddings**: Pinecone hosted model configuration is not migrated. The CLI migrates stored vector values. Use LambdaDB managed embeddings when you want LambdaDB to generate embeddings from source text after migration.
* **Hybrid shape**: Pinecone supports multiple hybrid patterns. Validate whether your workload uses a single dense index with `sparse_values`, separate dense and sparse indexes, or a document-schema index before choosing the target LambdaDB schema.
* **Bulk writes**: Regular upsert accepts request payloads up to 6 MB. Bulk upsert accepts up to 200 MB, but not for collections with managed embeddings.
* **Consistency**: Pinecone is eventually consistent for freshly written data. LambdaDB uses eventual reads by default, but supports `consistentRead` for strong read-after-write checks. The CLI validation uses strongly consistent sample fetches; use query overlap checks before cutover.
## Next steps
Review the shared LambdaDB Migration CLI workflow.
Define LambdaDB index configurations for your migrated data.
Rewrite Pinecone sparse-vector searches as LambdaDB sparse vector queries.
Combine lexical, dense vector, and sparse vector search.
# Migrate from Qdrant
Source: https://docs.lambdadb.ai/guides/migrations/qdrant/migrate-from-qdrant
Use the LambdaDB Migration CLI to move Qdrant collections, points, vectors, sparse vectors, and payload indexes to LambdaDB.
Use the LambdaDB Migration CLI to migrate Qdrant collections into LambdaDB. The CLI inventories your Qdrant collection, generates an editable LambdaDB mapping, creates the target LambdaDB collection when needed, streams points as LambdaDB documents, saves local checkpoints, and validates migrated records before cutover.
Qdrant models searchable records as **points** made of an `id`, one or more vectors, and an optional `payload`. LambdaDB models records as **documents**: each document is stored as JSON, while searchable fields are declared in the collection's `indexConfigs`.
## What the CLI supports
| Qdrant data | LambdaDB target | Migration behavior |
| :------------------- | :------------------- | :--------------------------------------------------------------------------------------------------------------- |
| Collection | Collection | Keep one LambdaDB collection per Qdrant collection unless you are also changing tenancy or isolation boundaries. |
| Point ID | Document `id` | Numeric Qdrant IDs are converted to strings. UUID IDs remain the same string value. |
| Unnamed dense vector | `vector` field | The generated mapping uses `dense` as the default target field. |
| Named dense vector | `vector` field | Each Qdrant named vector maps to its own LambdaDB vector field. |
| Sparse vector | `sparseVector` field | Qdrant `indices` and `values` arrays are converted to a LambdaDB sparse object. |
| Payload | Document fields | Payload fields are flattened by default. Indexed payload fields become LambdaDB index configs. |
| Payload index | `indexConfigs` field | Supported types include `keyword`, `text`, `long`, `double`, `boolean`, `datetime`, and `object`. |
Qdrant multi-vectors and Manhattan distance collections require workload-specific review. The CLI detects them and rejects the default migration path instead of silently creating a misleading LambdaDB schema.
## Step 1: Install the CLI
Install the latest release:
```bash theme={null}
curl -fsSL https://raw.githubusercontent.com/lambdadb/lambdadb-migration/main/install.sh | sh
```
Install a specific version:
```bash theme={null}
curl -fsSLO https://raw.githubusercontent.com/lambdadb/lambdadb-migration/main/install.sh
sh install.sh --version v0.1.5 --install-dir "$HOME/.local/bin"
```
Check the Qdrant command:
```bash theme={null}
lambdadb-migration qdrant --help
```
## Step 2: Set credentials
LambdaDB Cloud uses region-specific API base URLs. Use the **base URL**, **project name**, and **project API key** shown for your project in the LambdaDB Cloud console. Do not assume a global default URL or a fixed project name.
Set your LambdaDB connection values:
```bash theme={null}
export LAMBDADB_BASE_URL="YOUR_REGION_BASE_URL"
export LAMBDADB_PROJECT_NAME="YOUR_PROJECT_NAME"
export LAMBDADB_PROJECT_API_KEY="YOUR_PROJECT_API_KEY"
```
If your Qdrant deployment requires an API key, pass it with `--qdrant.api-key`.
## Step 3: Generate inventory and mapping
Run the inventory command against the Qdrant gRPC endpoint:
```bash theme={null}
lambdadb-migration inventory qdrant \
--qdrant.url http://localhost:6334 \
--qdrant.collection articles \
--output qdrant-inventory.yaml
```
The output includes the source inventory and an editable LambdaDB mapping:
```yaml theme={null}
mapping:
target:
collection: articles
createCollection: true
vectors:
dense:
targetField: dense
dimensions: 1536
similarity: cosine
sparseVectors:
sparse:
targetField: sparse
payload:
mode: flatten
rename:
metadata.url: metadata_url
indexConfigs:
tenant_id:
type: keyword
title:
type: text
metadata_url:
type: keyword
ids:
targetField: id
```
Review the generated mapping before running the migration. In particular, check vector dimensions, vector similarities, payload index types, renamed dotted fields, and the target collection name.
Generated mappings set `target.createCollection: true` by default. With that setting, the migration creates the LambdaDB collection if it is missing and waits until it is ready before writing documents.
## Step 4: Run a dry run
Use a dry run to validate the mapping and inspect the planned migration without writing documents:
```bash theme={null}
lambdadb-migration qdrant \
--qdrant.url http://localhost:6334 \
--qdrant.collection articles \
--lambdadb.base-url "$LAMBDADB_BASE_URL" \
--lambdadb.project-name "$LAMBDADB_PROJECT_NAME" \
--lambdadb.api-key "$LAMBDADB_PROJECT_API_KEY" \
--lambdadb.collection articles \
--mapping-file qdrant-inventory.yaml \
--migration.dry-run
```
## Step 5: Run the migration
Run the migration with validation enabled:
```bash theme={null}
lambdadb-migration qdrant \
--qdrant.url http://localhost:6334 \
--qdrant.collection articles \
--lambdadb.base-url "$LAMBDADB_BASE_URL" \
--lambdadb.project-name "$LAMBDADB_PROJECT_NAME" \
--lambdadb.api-key "$LAMBDADB_PROJECT_API_KEY" \
--lambdadb.collection articles \
--mapping-file qdrant-inventory.yaml \
--migration.write-mode bulk \
--migration.validate \
--migration.validation-report validation-report.json
```
Migration progress is written to stderr with accepted count, percent, batch size, rate, and elapsed time.
The CLI stores checkpoints under `.lambdadb-migration/checkpoints` by default. If the command is interrupted, rerun the same command to resume. Use `--migration.restart` to ignore an existing checkpoint and start from the beginning.
Use `--migration.create-collection=false` when the target LambdaDB collection already exists and the migration should not create it.
## Step 6: Review validation
`--migration.validate` checks the accepted record count, fetches a sample of migrated documents from LambdaDB using strongly consistent reads, and compares sampled fields.
`--migration.validation-report` writes the validation result as JSON. The report includes source count, accepted records, LambdaDB `numDocs`, sampled IDs, compared sample count, query overlap results, and validation errors.
`numDocs` is reported for visibility, but sample fetch and field comparison are the stronger validation checks for read-after-write confirmation.
For dense-vector migrations, add query overlap checks:
```bash theme={null}
lambdadb-migration qdrant \
--qdrant.url http://localhost:6334 \
--qdrant.collection articles \
--lambdadb.base-url "$LAMBDADB_BASE_URL" \
--lambdadb.project-name "$LAMBDADB_PROJECT_NAME" \
--lambdadb.api-key "$LAMBDADB_PROJECT_API_KEY" \
--lambdadb.collection articles \
--mapping-file qdrant-inventory.yaml \
--migration.validate \
--migration.validation-report validation-report.json \
--migration.query-overlap
```
By default, `--migration.query-overlap` reports dense-vector overlap without failing the migration. Set `--migration.query-overlap-min-ratio` above `0` to require a minimum average overlap.
## Mapping details
Use this distance mapping when creating LambdaDB vector fields:
| Qdrant distance | LambdaDB similarity |
| :-------------- | :----------------------------------------------------------------------------- |
| `Cosine` | `cosine` |
| `Dot` | `dot_product` |
| `Euclid` | `euclidean` |
| `Manhattan` | No direct equivalent. Re-evaluate the embedding/search setup before migrating. |
For a Qdrant point with a single dense vector:
```json Qdrant point theme={null}
{
"id": 1,
"vector": [0.12, 0.34, 0.56],
"payload": {
"tenant_id": "acme",
"title": "Refund policy",
"body": "Refunds are available within 7 days.",
"created_at": "2026-05-01T10:00:00Z"
}
}
```
the CLI writes a LambdaDB document like this:
```json LambdaDB document theme={null}
{
"id": "1",
"tenant_id": "acme",
"title": "Refund policy",
"body": "Refunds are available within 7 days.",
"created_at": "2026-05-01T10:00:00Z",
"dense": [0.12, 0.34, 0.56]
}
```
For Qdrant named vectors:
```json Qdrant point with named vectors theme={null}
{
"id": 1,
"vector": {
"title_dense": [0.1, 0.2, 0.3],
"body_dense": [0.4, 0.5, 0.6]
},
"payload": {
"title": "Refund policy",
"body": "Refunds are available within 7 days."
}
}
```
the CLI writes separate LambdaDB vector fields:
```json LambdaDB document theme={null}
{
"id": "1",
"title": "Refund policy",
"body": "Refunds are available within 7 days.",
"title_dense": [0.1, 0.2, 0.3],
"body_dense": [0.4, 0.5, 0.6]
}
```
For Qdrant sparse vectors, the CLI converts `indices` and `values` to an object whose keys are index strings:
```json Qdrant sparse vector theme={null}
{
"indices": [1, 42],
"values": [0.22, 0.8]
}
```
```json LambdaDB sparse vector theme={null}
{
"sparse": {
"1": 0.22,
"42": 0.8
}
}
```
## Reduce application changes with SDK compatibility
After data is in LambdaDB, Python and TypeScript applications can use LambdaDB's Qdrant compatibility clients as a bridge before rewriting every query to native LambdaDB APIs.
The compatibility clients support common Qdrant-style calls such as collection creation, dense-vector upsert, dense-vector query, retrieve, delete, filtered scroll, and unfiltered count. They are not full Qdrant client replacements, and unsupported behavior raises an explicit error where possible.
Use Python `lambdadb.compat.qdrant` or TypeScript `@functional-systems/lambdadb/compat/qdrant` to reduce application code changes.
## Rewrite vector search
A Qdrant query against a named vector:
```python Python theme={null}
from qdrant_client import models
qdrant.query_points(
collection_name="articles",
query=query_vector,
using="dense",
limit=10,
query_filter=models.Filter(
must=[
models.FieldCondition(
key="tenant_id",
match=models.MatchValue(value="acme"),
)
]
),
with_payload=True,
)
```
becomes a LambdaDB `knn` query:
```python Python theme={null}
results = coll.query(
query={
"knn": {
"field": "dense",
"queryVector": query_vector,
"k": 10,
"filter": {
"queryString": {
"query": '"acme"',
"defaultField": "tenant_id",
}
},
}
},
size=10,
)
```
When using LambdaDB managed embeddings, send query text instead of a vector:
```python Python theme={null}
results = coll.query(
query={
"knn": {
"field": "bodyEmbedding",
"queryText": "refund policy",
"k": 10,
}
},
size=10,
)
```
## Rewrite filters
Qdrant filters commonly use `must`, `must_not`, and `should` clauses. In LambdaDB, use a `bool` query when the filter logic is larger than a single query string.
```json Qdrant filter theme={null}
{
"must": [
{ "key": "tenant_id", "match": { "value": "acme" } },
{ "key": "created_at", "range": { "gte": "2026-01-01T00:00:00Z" } }
],
"must_not": [
{ "key": "status", "match": { "value": "deleted" } }
]
}
```
```json LambdaDB bool query theme={null}
{
"bool": [
{
"queryString": {
"query": "tenant_id:acme"
},
"occur": "filter"
},
{
"queryString": {
"query": "created_at:[2026-01-01T00:00:00Z TO *]"
},
"occur": "filter"
},
{
"queryString": {
"query": "status:deleted"
},
"occur": "must_not"
}
]
}
```
Use field types intentionally:
| Qdrant payload use | LambdaDB index type |
| :---------------------------------------- | :------------------ |
| Exact string match, tags, IDs, tenant IDs | `keyword` |
| Natural-language matching | `text` |
| Integer range or sorting | `long` |
| Floating-point range or sorting | `double` |
| Date/time range or sorting | `datetime` |
| Boolean flags | `boolean` |
| Nested JSON kept as a searchable object | `object` |
## Rewrite hybrid search
Qdrant hybrid queries often use `prefetch` plus RRF fusion across dense and sparse vectors.
```json Qdrant hybrid query theme={null}
{
"prefetch": [
{
"query": { "indices": [1, 42], "values": [0.22, 0.8] },
"using": "sparse",
"limit": 20
},
{
"query": [0.01, 0.45, 0.67],
"using": "dense",
"limit": 20
}
],
"query": { "fusion": "rrf" },
"limit": 10
}
```
In LambdaDB, express the same dense/sparse fusion with `rrf`:
```json LambdaDB hybrid query theme={null}
{
"rrf": [
{
"sparseVector": {
"field": "sparse",
"queryVector": {
"1": 0.22,
"42": 0.8
}
}
},
{
"knn": {
"field": "dense",
"queryVector": [0.01, 0.45, 0.67],
"k": 20
}
}
]
}
```
You can also combine lexical and vector relevance:
```json LambdaDB lexical + vector hybrid query theme={null}
{
"l2": [
{
"queryString": {
"query": "refund policy",
"defaultField": "body"
},
"boost": 0.4
},
{
"knn": {
"field": "dense",
"queryVector": [0.01, 0.45, 0.67],
"k": 20
},
"boost": 0.6
}
]
}
```
## Common gotchas
* **IDs**: LambdaDB document IDs are strings. Numeric Qdrant IDs are converted to strings during migration.
* **Field names**: LambdaDB field names cannot contain dots. The generated mapping renames dotted Qdrant payload keys, such as `metadata.url` to `metadata_url`.
* **Schema**: Qdrant can store payload without deciding every searchable field up front. In LambdaDB, decide which fields need indexes before migration.
* **Bulk writes**: Regular upsert accepts request payloads up to 6 MB. Bulk upsert accepts up to 200 MB, but not for collections with managed embeddings.
* **Sparse vectors**: Qdrant sparse vectors use separate `indices` and `values`; LambdaDB uses object key-value pairs.
* **Multi-vectors**: Qdrant multi-vectors store matrices for late-interaction models. Plan and validate these workloads separately.
* **Consistency**: LambdaDB uses eventual reads by default, but supports `consistentRead` for strong read-after-write checks. The CLI validation uses strongly consistent sample fetches. For bulk upsert, allow time for documents to become visible after indexing completes.
## Next steps
Review the shared LambdaDB Migration CLI workflow.
Keep common Qdrant-style Python and TypeScript client calls while migrating application code.
Define LambdaDB index configurations for your migrated data.
Load larger migrated datasets through the bulk upsert workflow.
Combine lexical, dense vector, and sparse vector search.
# Qdrant SDK compatibility
Source: https://docs.lambdadb.ai/guides/migrations/qdrant/qdrant-sdk-compatibility
Use LambdaDB's Python and TypeScript Qdrant compatibility clients to reduce application code changes during Qdrant migrations.
LambdaDB provides explicit Qdrant-style compatibility clients for Python and TypeScript applications. Use them when you have already moved or plan to move Qdrant data into LambdaDB, but want to keep common Qdrant client calls while you migrate application code.
The compatibility clients are not full Qdrant client replacements. They cover the common dense-vector search and RAG subset, map Qdrant points to LambdaDB documents, and raise an explicit unsupported-feature error for behavior that LambdaDB cannot safely emulate.
## When to use this path
Use the Qdrant compatibility clients when:
* Your application already uses Qdrant's Python or JavaScript/TypeScript SDK.
* You want a smaller application change than rewriting all query code to LambdaDB native APIs at once.
* Your workload uses dense vectors, named dense vectors, payload filters, document retrieval, deletes, scroll, or collection metadata checks.
Use LambdaDB native APIs directly when:
* You are building new application code.
* You need LambdaDB-specific query features such as lexical plus vector hybrid search.
* You want the clearest long-term API surface after migration.
## Install
Python support is available in `lambdadb >= 0.8.2`:
```bash theme={null}
pip install "lambdadb>=0.8.2"
```
TypeScript support is available in `@functional-systems/lambdadb >= 0.4.3`:
```bash theme={null}
npm install @functional-systems/lambdadb
```
## Connection settings
LambdaDB Cloud uses region-specific API base URLs. Use the **base URL**, **project name**, and **project API key** shown for your project in the LambdaDB Cloud console. Do not assume a global default URL or a fixed project name.
Pass those values to the compatibility client instead of a Qdrant URL.
## Python
Change the import and client construction explicitly:
```diff theme={null}
- from qdrant_client import QdrantClient, models
+ from lambdadb.compat.qdrant import QdrantCompatClient as QdrantClient, models
- client = QdrantClient(url="http://localhost:6333")
+ client = QdrantClient(
+ project_api_key="",
+ base_url="",
+ project_name="",
+ )
```
Create a collection, write points, and query with Qdrant-style calls:
```python Python theme={null}
from lambdadb.compat.qdrant import QdrantCompatClient, models
client = QdrantCompatClient(
project_api_key="",
base_url="",
project_name="",
)
client.create_collection(
collection_name="articles",
vectors_config=models.VectorParams(
size=3,
distance=models.Distance.COSINE,
),
payload_schema={
"tenant_id": models.PayloadSchemaType.KEYWORD,
},
)
client.upsert(
collection_name="articles",
points=[
models.PointStruct(
id=1,
vector=[1.0, 0.0, 0.0],
payload={"tenant_id": "acme", "title": "Refund policy"},
)
],
)
results = client.query_points(
collection_name="articles",
query=[1.0, 0.0, 0.0],
query_filter=models.Filter(
must=[
models.FieldCondition(
key="tenant_id",
match=models.MatchValue(value="acme"),
)
]
),
limit=10,
with_payload=True,
with_vectors=False,
)
```
## TypeScript
Change the import and client construction explicitly:
```diff theme={null}
- import { QdrantClient, models } from "@qdrant/js-client-rest";
+ import { QdrantCompatClient as QdrantClient, models } from "@functional-systems/lambdadb/compat/qdrant";
- const client = new QdrantClient({ url: "http://localhost:6333" });
+ const client = new QdrantClient({
+ projectApiKey: "",
+ baseUrl: "",
+ projectName: "",
+ });
```
Create a collection, write points, and query with Qdrant-style calls:
```ts TypeScript theme={null}
import { QdrantCompatClient, models } from "@functional-systems/lambdadb/compat/qdrant";
const client = new QdrantCompatClient({
projectApiKey: "",
baseUrl: "",
projectName: "",
});
await client.createCollection("articles", {
vectorsConfig: new models.VectorParams({
size: 3,
distance: models.Distance.COSINE,
}),
payloadSchema: {
tenant_id: models.PayloadSchemaType.KEYWORD,
},
});
await client.upsert("articles", {
points: [
new models.PointStruct({
id: 1,
vector: [1.0, 0.0, 0.0],
payload: { tenant_id: "acme", title: "Refund policy" },
}),
],
});
const results = await client.queryPoints("articles", {
query: [1.0, 0.0, 0.0],
queryFilter: new models.Filter({
must: [
new models.FieldCondition({
key: "tenant_id",
match: new models.MatchValue({ value: "acme" }),
}),
],
}),
limit: 10,
withPayload: true,
withVectors: false,
});
```
## Supported Qdrant-style APIs
| Area | Python | TypeScript | Notes |
| :------------------ | :----------------------- | :------------------------------------------------ | :------------------------------------------------------------------------------------------------------------------------------------------- |
| Client | `QdrantCompatClient` | `QdrantCompatClient` | Accepts LambdaDB connection settings or an existing LambdaDB client. |
| Collection exists | `collection_exists()` | `collectionExists()` / `collection_exists()` | Maps to LambdaDB collection metadata lookup. |
| Collection metadata | `get_collection()` | `getCollection()` / `get_collection()` | Returns minimal Qdrant-style vector config used by integrations. |
| Create collection | `create_collection()` | `createCollection()` / `create_collection()` | Supports dense vectors and named dense vectors. Use payload schema for filter fields. |
| Recreate collection | `recreate_collection()` | `recreateCollection()` / `recreate_collection()` | Deletes the collection if it exists, then creates it. |
| Delete collection | `delete_collection()` | `deleteCollection()` / `delete_collection()` | Maps to LambdaDB collection delete. |
| Payload index | `create_payload_index()` | `createPayloadIndex()` / `create_payload_index()` | Limited to empty collections unless the same index already exists. Prefer declaring payload schema at creation time. |
| Upsert points | `upsert()` | `upsert()` | Dense vectors only. Qdrant point IDs become LambdaDB document IDs. |
| Upload points | `upload_points()` | `uploadPoints()` / `upload_points()` | Batches points through upsert. |
| Upload collection | `upload_collection()` | `uploadCollection()` / `upload_collection()` | Converts vectors, IDs, and payload arrays into points. |
| Query | `query_points()` | `queryPoints()` / `query_points()` | Dense vector query plus supported payload filters. |
| Query alias | `search()` | `query()` / `search()` | TypeScript `query()` supports Qdrant JS-style `filter`, `with_payload`, and `with_vector`. |
| Retrieve | `retrieve()` | `retrieve()` | Uses strongly consistent LambdaDB fetches. |
| Delete points | `delete()` | `delete()` | Supports point IDs and supported Qdrant filters. |
| Scroll | `scroll()` | `scroll()` | Supports filtered scroll with payload and vector response selectors. Returned offsets are LambdaDB page tokens, not Qdrant point-id offsets. |
| Count | `count()` | `count()` | Unfiltered collection count only. |
## Data mapping
| Qdrant concept | LambdaDB mapping |
| :------------------------- | :------------------------------------ |
| Point ID | Document `id`, stringified |
| Original numeric ID | Reserved `_qdrant_id` field |
| Unnamed dense vector | Reserved `_qdrant_vector` field |
| Named dense vector `title` | Reserved `_qdrant_vector_title` field |
| Payload fields | Top-level document fields |
Payload fields cannot use `id` or the reserved `_qdrant_` prefix.
## Payload and vector selectors
Boolean selectors and field-list selectors are supported on query and retrieve paths. Field-list payload selectors are mapped to LambdaDB `fields.include` where possible and are also applied to the Qdrant-style response payload.
Vector-name selectors request vector values from LambdaDB and filter the returned Qdrant-style vector object by Qdrant vector name:
```python Python theme={null}
client.query_points(
collection_name="articles",
query=[1.0, 0.0, 0.0],
with_vectors=["title"],
)
```
```ts TypeScript theme={null}
await client.query("articles", {
query: [1.0, 0.0, 0.0],
with_vector: ["title"],
});
```
`scroll()` maps to LambdaDB list documents. Filters are translated to LambdaDB list filters before pagination. When Qdrant-style vector selectors are requested, the compatibility layer requests vector values from LambdaDB and filters the returned vector object while shaping the response.
Returned `next_offset` / `nextOffset` values are LambdaDB page tokens. Pass the returned token back as the next `offset`; numeric Qdrant point-ID offsets are not supported.
```python Python theme={null}
records, next_offset = client.scroll(
collection_name="articles",
scroll_filter=models.Filter(
must=[
models.FieldCondition(
key="tenant_id",
match=models.MatchValue(value="acme"),
)
]
),
with_payload=["tenant_id"],
with_vectors=["title"],
)
```
```ts TypeScript theme={null}
const [records, nextOffset] = await client.scroll("articles", {
scrollFilter: new models.Filter({
must: [
new models.FieldCondition({
key: "tenant_id",
match: new models.MatchValue({ value: "acme" }),
}),
],
}),
withPayload: ["tenant_id"],
withVectors: ["title"],
});
```
## Filter support
| Qdrant filter | Status |
| :--------------------------------- | :--------------------------------------------------------------------------------- |
| `Filter.must` | Supported |
| `Filter.should` | Supported through LambdaDB bool clauses |
| `Filter.must_not` / `mustNot` | Supported |
| `FieldCondition.match=MatchValue` | Supported |
| `FieldCondition.match=MatchAny` | Supported |
| `FieldCondition.match=MatchExcept` | Supported |
| `FieldCondition.range` | Supported |
| `HasIdCondition` | Supported |
| `MatchText` | Supported for whitespace-separated text terms through LambdaDB text query strings. |
| Geo filters | Unsupported |
| Nested object filters | Unsupported |
## Unsupported behavior
* Local Qdrant mode, including `path` and `location=":memory:"`
* Sparse vector upsert through the compatibility client
* Multi-vector comparators
* Geo payload indexes and geo filters
* Filtered count
* Query offset
* `score_threshold`
* HNSW and search tuning semantics beyond warnings
## How this fits with the Migration CLI
The LambdaDB Migration CLI moves data from Qdrant to LambdaDB. The Qdrant compatibility clients help reduce application code changes after data is in LambdaDB.
For a full Qdrant migration workflow, start with the migration guide:
Move Qdrant collections, points, vectors, sparse vectors, and payload indexes to LambdaDB.
Install and use the official LambdaDB SDKs.
# Boolean query
Source: https://docs.lambdadb.ai/guides/search/boolean
Combine multiple sub-queries in LambdaDB using boolean logic with filter, must, must_not, and should clauses. Supports boost scoring for relevance.
A query that matches documents based on boolean combinations of other queries.
The boolean query maps to [Lucene BooleanQuery](https://lucene.apache.org/core/10_4_0/core/org/apache/lucene/search/BooleanQuery.html).
## Boolean query array parameters
When used within a bool query array, each object can contain:
| Parameter | Description | Type | Required | Default |
| :-------- | :----------------------------- | :----- | :------- | :------ |
| query | Query string object | object | ✓ | |
| occur | Boolean occurrence type | string | | should |
| boost | Score multiplier for relevance | float | | 1.0 |
It is built using one or more boolean clauses, each clause with a typed occurrence. The occurrence types are:
### Occur
| Occur | Description |
| :-------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| filter | The clause (query) must appear in matching documents. However, unlike `must`, the score of the query will be ignored. Each query defined under a `filter` acts as a logical "AND", returning only documents that match all the specified queries. |
| must | The clause (query) must appear in matching documents and will contribute to the score. Each query defined under a `must` acts as a logical "AND", returning only documents that match all the specified queries. |
| must\_not | The clause (query) must not appear in the matching documents. Each query defined under a `must_not` acts as a logical "NOT", returning only documents that do not match any of the specified queries. |
| should | The clause (query) should appear in the matching document. Each query defined under a `should` acts as a logical "OR", returning documents that match any of the specified queries. |
### Boost
Boost values that are less than 1.0 will give less importance to this query compared to other ones
while values that are greater than 1.0 will give more importance to the scores returned by this query.
## Examples
### Boolean query with occur
```json theme={null}
{
"bool": [
{
"queryString": {
"query": "node_type:NODE"
},
"occur" : "filter"
},
{
"queryString": {
"query": "content:LambdaDB"
},
"occur" : "should"
}
]
}
```
This query will return top documents that are both of type `NODE` and have the highest score value.
### Boolean query with boost
The boost value must be greater than zero.
```json theme={null}
{
"bool": [
{
"queryString": {
"query": "node_type:NODE"
},
"boost": 0.8
},
{
"queryString": {
"query": "content:LambdaDB"
},
"boost": 0.2
}
]
}
```
The final score of the matched document is calculated as proportional to the boost values.
# Hybrid query
Source: https://docs.lambdadb.ai/guides/search/hybrid
Combine vector and lexical search in LambdaDB for better relevance. Choose from RRF, Min-Max, or L2 norm rescoring to normalize hybrid results.
A hybrid query combines vector search with lexical search to achieve better relevance by leveraging both semantic similarity and keyword matching.
Full-text search and vector search use fundamentally different scoring mechanisms - text search typically uses relevance scores based on term frequency and document frequency, while vector search uses similarity distances. Without proper normalization, one search method may dominate the results, leading to suboptimal ranking. Hybrid queries address this by combining and normalizing scores from both methods.
LambdaDB supports three rescoring methods to combine results from different query types: `rrf` (Reciprocal Rank Fusion), `mm` (min-max), and `l2` (l2\_norm).
* **RRF** combines rankings by taking the reciprocal of each result's rank position, providing balanced weighting across different search methods.
* **MinMax** normalization scales scores to a 0-1 range before combining.
* **L2 norm** uses Euclidean distance-based normalization to merge relevance scores from multiple query sources.
Regardless of the rescoring method used, the final combined score is always normalized to a value between 0 and 1.
## Parameters
### Rescoring methods
| Method | Description | Boost support |
| :----- | :---------------------------------------------------- | :------------ |
| rrf | Reciprocal Rank Fusion - balanced ranking combination | No |
| mm | Min-Max normalization scaling | Yes |
| l2 | Euclidean distance-based normalization | Yes |
### Query object parameters
Each query object within the rescoring method array can contain:
| Parameter | Description | Type | Required | Default |
| :-------- | :----------------------------------------------- | :----- | :------- | :------ |
| query | Query object (queryString/knn/sparseVector/bool) | object | ✓ | |
| boost | Score multiplier for relevance | float | | 0.5 |
A hybrid query can include up to two query objects. If you need to express more complex logic within a query object, use a `boolean query` to combine multiple conditions.
### Boost constraints
* The boost parameter is only available for `mm` and `l2` rescoring methods.
* The sum of all boost values must equal 1.0.
* Each individual boost value must be between 0 and 1.
## Examples
### L2-norm hybrid query
```json theme={null}
{
"l2": [
{
"queryString": {
"query": "content:LambdaDB"
},
"boost": 0.7
},
{
"knn": {
"filter": {
"queryString": {
"query": "\"https://lambdadb.ai\"",
"defaultField": "metadata.url"
}
},
"field": "text_embedding",
"queryVector": [0.1, 0.2, 0.3],
"k": 5
},
"boost": 0.3
}
]
}
```
This example combines:
* A `queryString` query with 0.7 boost weight.
* A `knn` vector query with pre-filtering and 0.3 boost weight.
* Uses L2-norm rescoring to merge the results.
### MinMax hybrid query with sparse vector
```json theme={null}
{
"mm": [
{
"sparseVector": {
"field": "sparse_embedding",
"queryVector": {
"machine": 0.8,
"learning": 0.6,
"AI": 0.4
}
},
"boost": 0.6
},
{
"knn": {
"field": "dense_embedding",
"queryVector": [0.1, 0.2, 0.3],
"k": 10
},
"boost": 0.4
}
]
}
```
This example combines:
* A `sparseVector` query with 0.6 boost weight.
* A `knn` dense vector query with 0.4 boost weight.
* Uses MinMax normalization to merge the results.
### RRF hybrid query
```json theme={null}
{
"rrf": [
{
"queryString": {
"query": "machine learning",
"defaultField": "content"
}
},
{
"knn": {
"field": "content_embedding",
"queryVector": [0.1, 0.2, 0.3],
"k": 10
}
}
]
}
```
This example uses Reciprocal Rank Fusion to balance text and vector search results.
The final returned documents may not include the requested number of documents from the `knn` query. This is because the scores of documents returned solely from other queries may be higher than those of the top k documents returned from the `knn`.
# Query limits
Source: https://docs.lambdadb.ai/guides/search/limits
Review LambdaDB query constraints including maximum result size, execution timeout, boolean clause limits, and sparse vector value caps per request.
| Metric | Limit |
| :------------------------------------------ | :---- |
| Max `size` value | 100 |
| Max result size | 6MB |
| Max execution time | 30s |
| Max boolean clauses | 4,096 |
| Max non-zero values for sparse vector query | 4,096 |
| Value range for sparse vector query | 0-64 |
The query result size is affected by the dimension of the vectors and
whether vector values are included in the result.
If a query fails due to exceeding the 6MB result size limit, choose a lower `size` value,
or `includeVectors=false` (Python: `include_vectors=False`) to exclude vector values from the result.
# Query string search
Source: https://docs.lambdadb.ai/guides/search/query-string
Search LambdaDB documents with Apache Lucene query string syntax, including proximity search, regular expressions, fuzzy matching, term boosting, minimum-should-match, and interval functions.
LambdaDB supports a simple string query that uses the [Lucene Query Syntax](https://lucene.apache.org/core/10_4_0/queryparser/org/apache/lucene/queryparser/flexible/standard/StandardQueryParser.html) to search documents.
This gives `queryString` access to expressive lexical search features such as terms, phrases, wildcards, boolean operators, range operators, proximity search, regular expressions, fuzzy term matching, term boosting, minimum-should-match constraints, interval functions, and special character escaping.
## Supported syntax highlights
| Feature | Example | Use it to |
| :------------------------- | :------------------------------------------- | :--------------------------------------------------------- |
| Term search | `hello` | Match documents containing a term. |
| Phrase search | `"hello world"` | Match an exact phrase. |
| Field search | `title:LambdaDB` | Target a specific field. |
| Boolean logic and grouping | `title:database AND (serverless OR managed)` | Combine required, optional, and excluded clauses. |
| Wildcards | `vector*` | Match terms by prefix or pattern. |
| Proximity search | `"serverless database"~4` | Match words that occur near each other. |
| Regular expression | `/vec(tor\|tors)/` | Match terms using a regular expression. |
| Fuzzy term matching | `lambdadb~2` | Match terms within an edit distance. |
| Range query | `created_at:[2024-01-01T00:00:00Z TO *]` | Match numeric, datetime, or keyword ranges. |
| Term boosting | `database^2 OR storage^0.5` | Increase or decrease the relevance weight of clauses. |
| Minimum should match | `(semantic vector keyword)@2` | Require at least N optional clauses to match. |
| Interval function | `fn:ordered(serverless vector database)` | Express ordered or positional relationships between terms. |
## Parameters
| Parameter | Description | Type | Required | Default |
| :----------- | :--------------------------------------- | :------ | :------- | :------ |
| query | Query string | string | ✓ | |
| defaultField | Default field name to apply query | string | | |
| skipSyntax | Skip syntax check for special characters | boolean | | false |
A query to a collection with `object` field type requires special handling.
## Examples
### Basic query
```json theme={null}
{
"queryString": {
"query": "hello world",
"defaultField": "text"
}
}
```
When multiple terms are separated by spaces, each term is treated as an optional clause on the `defaultField`.
The above example is equivalent to:
```json theme={null}
{
"queryString": {
"query": "text:hello OR text:world"
}
}
```
You can also query multiple fields by specifying field names in the query string. Terms are separated by spaces, and specific fields are targeted using the `field:value` syntax:
```json theme={null}
{
"queryString": {
"query": "hello world keyword:python",
"defaultField": "text"
}
}
```
### Exact keyword match
Use a `keyword` field when you want to match a literal value instead of analyzed text.
```json theme={null}
{
"queryString": {
"query": "python",
"defaultField": "keyword"
}
}
```
If the keyword contains spaces or query syntax characters, wrap the value in double quotes (`"`):
```json theme={null}
{
"queryString": {
"query": "\"hello world\"",
"defaultField": "keyword"
}
}
```
For raw literal values that include special characters, you can also use the `skipSyntax` option:
```json theme={null}
{
"queryString": {
"query": "https://sample_url.com",
"defaultField": "keyword",
"skipSyntax": true
}
}
```
Alternatively, quote the literal value:
```json theme={null}
{
"queryString": {
"query": "\"https://sample_url.com\"",
"defaultField": "keyword"
}
}
```
### Phrase search
On analyzed text fields, wrap terms in double quotes (`"`) to search for an exact phrase.
```json theme={null}
{
"queryString": {
"query": "\"serverless database\"",
"defaultField": "content"
}
}
```
### Proximity search
Use proximity search to match phrase terms that appear near each other, even when they are not adjacent.
The number after `~` controls the maximum distance between phrase terms.
```json theme={null}
{
"queryString": {
"query": "\"serverless database\"~4",
"defaultField": "content"
}
}
```
### Regular expression query
Wrap a regular expression in forward slashes (`/`) to match terms by pattern.
```json theme={null}
{
"queryString": {
"query": "/vec(tor|tors)/",
"defaultField": "content"
}
}
```
### Fuzzy term matching
Use `~` after a term to match terms within an edit distance.
This is useful for typos, spelling variants, and noisy text.
```json theme={null}
{
"queryString": {
"query": "lambdadb~2",
"defaultField": "content"
}
}
```
### Term boosting
Use `^` to increase or decrease the relevance weight of a term, phrase, range expression, or grouped clause.
```json theme={null}
{
"queryString": {
"query": "database^2 OR storage^0.5",
"defaultField": "content"
}
}
```
You can also boost a grouped sub-query:
```json theme={null}
{
"queryString": {
"query": "title:(serverless OR managed)^2.5 OR content:database"
}
}
```
### Minimum should match
Use the minimum-should-match operator (`@`) on a disjunction group to require at least the specified number of optional clauses to match.
```json theme={null}
{
"queryString": {
"query": "(semantic vector keyword)@2",
"defaultField": "content"
}
}
```
### Interval function
Use interval functions with the `fn:` prefix to express ordered or positional relationships between terms.
```json theme={null}
{
"queryString": {
"query": "fn:ordered(serverless vector database)",
"defaultField": "content"
}
}
```
You can target a field and combine interval functions:
```json theme={null}
{
"queryString": {
"query": "title:fn:maxwidth(5 fn:atLeast(2 serverless vector database))"
}
}
```
### Range query
Range queries allow you to search for values within a specific range using range query syntax.
#### Supported field types
Range queries work with the following field types:
* `long`
* `double`
* `datetime`
* `keyword`
#### Syntax
* **Inclusive range**: `[min TO max]` includes both lower and upper bounds.
* **Exclusive range**: `{min TO max}` excludes both lower and upper bounds.
```json theme={null}
{
"queryString": {
"query": "[123 TO 456]",
"defaultField": "long"
}
}
```
```json theme={null}
{
"queryString": {
"query": "{2024-03-10T09:15:45Z TO 2024-03-10T09:15:46Z}",
"defaultField": "datetime"
}
}
```
# Search overview
Source: https://docs.lambdadb.ai/guides/search/search-overview
Learn how LambdaDB search works, including query types, common parameters like size and consistency, field selection, sorting, and partition filtering.
A search query, or query, is a request for information about data in LambdaDB collections.
LambdaDB supports several search methods:
* Search for exact values: search for exact values or ranges of numbers, dates, IPs, or strings.
* Full-text search: use full text queries to query unstructured textual data and find documents that best match query terms.
* Vector search: store vectors in LambdaDB and use approximate nearest neighbor (ANN) to find vectors that are similar, supporting use cases like semantic search.
If you use managed embedding vector fields, see [Managed embeddings](/guides/collections/managed-embeddings) for the current supported providers and models.
Common parameters for the request body of a query are as follows:
| Parameter | Description | Type | Required | Default |
| :-------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-------- | :------- | :------ |
| size | The number of results to return for each query | integer | ✓ | |
| query | Query object (details in the [subsections](/guides/search/query-string)) | object | ✓ | |
| includeVectors | Indicates whether vector values are included in the response | boolean | | false |
| consistentRead | Determines the read consistency model: If set to true, then the operation uses strongly consistent reads; otherwise, the operation uses eventually consistent reads. | boolean | | false |
| fields | A list of field names to include and/or exclude in the result. Use dot notation (e.g., user.name) to specify nested fields. | object | | |
| sort | Specifies the sorting criteria for the results (details in the [subsection](/guides/search/sort)) | object\[] | | |
| partitionFilter | Partition filter | object | | |
Depending on your data and your query, you may get fewer than `size` results.
This happens when `size` is larger than the number of possible matching documents for your query.
Setting `includeVectors` (or `include_vectors` in Python) to `true` will increase response size significantly, especially for high-dimensional vectors.
Use this option only when vector data is specifically needed for your application.
`include` is applied first, and then `exclude` is applied to the included fields when you set both in the `fields` parameter.
LambdaDB is eventually consistent by default, so there can be a slight delay before new or changed documents are visible to queries.
If your application requires strong (read-after-write) consistency, set `consistentRead` (or `consistent_read` in Python) to `true` when querying data from a collection, at the expense of potential higher latency and cost.
Query results are returned in the following format:
| Field | Description | Type |
| -------- | ---------------------------------------------------- | --------- |
| took | Milliseconds it took LambdaDB to execute the request | long |
| maxScore | Highest returned document `score` | float |
| total | The total number of matching documents | long |
| docs | Contains returned documents and metadata | object\[] |
## Example
```python Python theme={null}
from lambdadb import LambdaDB
with LambdaDB(project_api_key="YOUR_API_KEY", base_url="YOUR_BASE_URL", project_name="YOUR_PROJECT_NAME") as client:
coll = client.collection("my_collection")
results = coll.query(query={"queryString": {"query": "*:*"}}, size=10)
# `results.docs` contains items (each item includes `doc` and `score`).
# `results.documents` contains document bodies only (no scores).
```
Python: `LambdaDB` supports context manager usage. `__enter__` returns the client, and `__exit__` calls `client.close()` (closing the SDK-owned HTTP client) and makes the client unusable after the `with` block. If you don't use `with`, call `client.close()` when you're done. If you pass a custom `client=`/`async_client=`, you own closing it.
```typescript TypeScript theme={null}
import { LambdaDBClient } from "@functional-systems/lambdadb";
const client = new LambdaDBClient({
projectApiKey: "YOUR_API_KEY",
baseUrl: "YOUR_BASE_URL",
projectName: "YOUR_PROJECT_NAME",
});
const results = await client.collection("my_collection").query({
query: { queryString: { query: "*:*" } },
size: 10,
});
```
```go Go theme={null}
package main
import (
"context"
"log"
lambdadb "github.com/lambdadb/go-lambdadb"
)
func main() {
ctx := context.Background()
client := lambdadb.New(
lambdadb.WithBaseURL("YOUR_BASE_URL"),
lambdadb.WithProjectName("YOUR_PROJECT_NAME"),
lambdadb.WithAPIKey("YOUR_API_KEY"),
)
coll := client.Collection("my_collection")
results, err := coll.Query(ctx, lambdadb.QueryCollectionInput{
Query: map[string]interface{}{"queryString": map[string]interface{}{"query": "*:*"}},
Size: 10,
})
if err != nil {
log.Fatal(err)
}
_ = results
}
```
The response will look like this:
```json theme={null}
{
"took": 76,
"maxScore": 1.0,
"total": 1,
"docs": [
{
"collection": "example_collection",
"score": 1.0,
"doc": {
"id": "33201222",
"url": "https://en.wikipedia.org/wiki/LambdaDB",
"title": "LambdaDB is awesome",
"text": null
}
}
],
"isDocsInline": true
}
```
Matched documents are ordered by similarity from most similar to least similar by default.
Similarity is expressed as a `score`, and it is calculated based on the `BM25 algorithm` for full-text search
and the configured `similarity metric` for vector search.
# Sort search results
Source: https://docs.lambdadb.ai/guides/search/sort
Sort LambdaDB query results by one or more fields in ascending or descending order. Supports long, double, datetime, and keyword field types.
LambdaDB allows you to add one or more sorts on specific fields. Each sort can be reversed as well. The sort is defined on a per field level.
Sorting can be done on the following data types:
* `long`
* `double`
* `datetime`
* `keyword`
## Parameters
| Parameter | Description | Type | Required | Values |
| :-------- | :-------------------- | :----- | :------- | :-------- |
| field | Field name to sort by | string | ✓ | - |
| order | Sort order direction | string | | asc, desc |
### Sort order options
| Value | Description |
| :---- | :------------------------ |
| asc | Sorts in ascending order |
| desc | Sorts in descending order |
## Usage
The sort parameter accepts an array of sort objects, allowing multiple field sorting with different orders.
```python Python theme={null}
from lambdadb import LambdaDB
with LambdaDB(
project_api_key="YOUR_API_KEY",
base_url="YOUR_BASE_URL",
project_name="YOUR_PROJECT_NAME",
) as client:
coll = client.collection(collection_name)
results = coll.query(
query={"queryString": {"query": "*:*"}},
size=10,
sort=[{"field1": "desc"}, {"field2": "asc"}],
)
```
```typescript TypeScript theme={null}
import { LambdaDBClient } from "@functional-systems/lambdadb";
const client = new LambdaDBClient({
projectApiKey: "YOUR_API_KEY",
baseUrl: "YOUR_BASE_URL",
projectName: "YOUR_PROJECT_NAME",
});
const collection = client.collection(collectionName);
const results = await collection.query({
query: { queryString: { query: "*:*" } },
size: 10,
sort: [{ field1: "desc" }, { field2: "asc" }],
});
```
```go Go theme={null}
package main
import (
"context"
"log"
lambdadb "github.com/lambdadb/go-lambdadb"
)
func main() {
ctx := context.Background()
client := lambdadb.New(
lambdadb.WithBaseURL("YOUR_BASE_URL"),
lambdadb.WithProjectName("YOUR_PROJECT_NAME"),
lambdadb.WithAPIKey("YOUR_API_KEY"),
)
coll := client.Collection(collectionName)
results, err := coll.Query(ctx, lambdadb.QueryCollectionInput{
Query: map[string]interface{}{"queryString": map[string]interface{}{"query": "*:*"}},
Size: 10,
Sort: []map[string]interface{}{{"field1": "desc"}, {"field2": "asc"}},
})
if err != nil {
log.Fatal(err)
}
_ = results
}
```
## Examples
### Single field sort
Sort by timestamp in descending order (newest first):
```json theme={null}
[{"timestamp": "desc"}]
```
### Multiple field sort
Sort by priority (descending) first, then by created\_at (ascending):
```json theme={null}
[{"priority": "desc"}, {"created_at": "asc"}]
```
When multiple sort fields are specified, the sorting is applied in the order they appear in the array. Documents are first sorted by the first field, then by the second field for documents with the same first field value, and so on.
# Sparse vector query
Source: https://docs.lambdadb.ai/guides/search/sparse-vector
Search LambdaDB using sparse vector representations from learned sparse retrieval models. Pass precalculated token-weight pairs for precise matching.
A sparse vector query executes search using sparse vector representations, typically generated by learned sparse retrieval models.
You must provide precalculated token-weight pairs as your query vectors, where each pair represents a term and its corresponding relevance score.
Currently, LambdaDB does not support built-in natural language processing models for automatic sparse vector generation.
## Parameters
| Parameter | Description | Type | Required |
| :---------- | :--------------------------------------------- | :----- | :------- |
| field | The name of the vector field to search against | string | ✓ |
| queryVector | Query vector as token-weight pairs | object | ✓ |
## Examples
### Token-based sparse vector query
```json theme={null}
{
"sparseVector": {
"field": "example_sparse_field",
"queryVector": {
"LambdaDB": 0.5,
"awesome": 0.3,
"AI": 0.2
}
}
}
```
### Index-based sparse vector query
If you prefer to use the index-based format with separate values and index arrays, you can specify index positions as keys in the queryVector object:
```json theme={null}
{
"sparseVector": {
"field": "example_sparse_field",
"queryVector": {
"10": 0.5,
"45": 0.3,
"234": 0.2
}
}
}
```
# Vector query
Source: https://docs.lambdadb.ai/guides/search/vector
Run k-nearest-neighbor vector search in LambdaDB. Specify query vectors or query text, set k, apply pre-filters, and combine multi-field vector queries.
A vector query finds the k nearest vectors to a query vector or query text, as measured by a similarity metric.
## Parameters
| Parameter | Description | Type | Required |
| :---------- | :-------------------------------------------------- | :------- | :--------------------- |
| field | The name of the vector field to search against | string | ✓ |
| queryVector | Query vector for unmanaged vector fields | float\[] | Conditionally required |
| queryText | Query text for managed embedding vector fields | string | Conditionally required |
| k | Number of nearest neighbors to return as top `docs` | integer | ✓ |
| filter | Query to filter the documents that can match | object | |
Provide exactly one of `queryVector` or `queryText`.
## Examples
### Simple vector query
```json theme={null}
{
"knn": {
"field": "example_vector_field",
"queryVector": [
0.030255454,
-0.058824085,
-0.065448694,
-0.03987034,
0.060786933,
-0.15469691,
-0.043918714,
0.057719983,
0.054530356,
0.007080819
],
"k": 5
}
}
```
### Managed embedding vector query
Use `queryText` when the target field is a managed embedding vector field.
Supported providers and models are listed in [Managed embeddings](/guides/collections/managed-embeddings).
```json theme={null}
{
"knn": {
"field": "bodyEmbedding",
"queryText": "refund policy",
"k": 5
}
}
```
### Multi-field vector query
You can search across multiple vector fields simultaneously by wrapping multiple kNN objects in a boolean query.
This is useful when you have different types of embeddings (e.g., text embedding and image embedding) and want to combine their results.
```json theme={null}
{
"bool": [
{
"knn": {
"field": "text_embedding",
"queryVector": [
0.030255454,
-0.058824085,
-0.065448694,
-0.03987034,
0.060786933,
-0.15469691,
-0.043918714,
0.057719983,
0.054530356,
0.007080819
],
"k": 10
}
},
{
"knn": {
"field": "image_embedding",
"queryVector": [
0.125434521,
-0.087654321,
0.045123789,
-0.156789012,
0.098765432,
0.034567890,
-0.123456789,
0.076543210,
-0.089012345,
0.112345678
],
"k": 10
}
}
]
}
```
### Vector query with filter query
An example vector query with a filter for better performance and relevance:
```json theme={null}
{
"knn": {
"filter" : {
"queryString": {
"query": "node_type:NODE AND \"https://example.com/books/5514276\"",
"defaultField": "metadata.url"
}
},
"field": "example_vector_field",
"queryVector": [
0.030255454,
-0.058824085,
-0.065448694,
-0.03987034,
0.060786933,
-0.15469691,
-0.043918714,
0.057719983,
0.054530356,
0.007080819
],
"k": 5
}
}
```
For managed embedding vector fields, `queryVector` is not supported. Query the field with `knn.queryText`, and let LambdaDB generate the query embedding with the field's configured embedding model.
# Configure a collection
Source: https://docs.lambdadb.ai/reference/api/endpoint/collection-configure
patch /collections/{collectionName}
Configure a collection.
# Create a collection
Source: https://docs.lambdadb.ai/reference/api/endpoint/collection-create
post /collections
Create a collection.
# Delete a collection
Source: https://docs.lambdadb.ai/reference/api/endpoint/collection-delete
delete /collections/{collectionName}
Delete an existing collection.
# Describe a collection
Source: https://docs.lambdadb.ai/reference/api/endpoint/collection-describe
get /collections/{collectionName}
Get metadata of an existing collection.
# List collections
Source: https://docs.lambdadb.ai/reference/api/endpoint/collection-list
get /collections
List all collections in an existing project.
# Bulk upsert documents
Source: https://docs.lambdadb.ai/reference/api/endpoint/document-bulk-upsert
post /collections/{collectionName}/docs/bulk-upsert
Bulk upsert documents into a collection. Note that the maximum supported object size is 200MB. Bulk upsert is not supported for collections with managed embedding vector fields.
# Delete documents
Source: https://docs.lambdadb.ai/reference/api/endpoint/document-delete
post /collections/{collectionName}/docs/delete
Delete documents by document IDs or query filter from a collection.
# Fetch documents
Source: https://docs.lambdadb.ai/reference/api/endpoint/document-fetch
post /collections/{collectionName}/docs/fetch
Lookup and return documents by document IDs from a collection.
# Get bulk upsert URL
Source: https://docs.lambdadb.ai/reference/api/endpoint/document-get-bulk-upsert-url
get /collections/{collectionName}/docs/bulk-upsert
Request required info to upload documents. Bulk upsert is not supported for collections with managed embedding vector fields.
# List documents
Source: https://docs.lambdadb.ai/reference/api/endpoint/document-list
get /collections/{collectionName}/docs
List documents in a collection. Vector values are excluded unless includeVectors is true.
# List documents extended
Source: https://docs.lambdadb.ai/reference/api/endpoint/document-list-extended
post /collections/{collectionName}/docs/list
List documents in a collection with optional filters, partition filtering, field selection, and vector inclusion.
# Query a collection
Source: https://docs.lambdadb.ai/reference/api/endpoint/document-query
post /collections/{collectionName}/query
Search a collection with a query and return the most similar documents.
# Update documents
Source: https://docs.lambdadb.ai/reference/api/endpoint/document-update
post /collections/{collectionName}/docs/update
Update documents in a collection. Note that the maximum supported payload size is 6MB.
# Upsert documents
Source: https://docs.lambdadb.ai/reference/api/endpoint/document-upsert
post /collections/{collectionName}/docs/upsert
Upsert documents into a collection. Note that the maximum supported payload size is 6MB.
# API reference
Source: https://docs.lambdadb.ai/reference/api/introduction
Explore the LambdaDB REST API for managing projects, collections, and documents. Authenticate with your project API key to get started.
# SDK reference
Source: https://docs.lambdadb.ai/reference/sdk/introduction
Install and use the official LambdaDB SDKs for Python, TypeScript, and Go. Each SDK wraps the REST API with idiomatic client libraries.
## Compatibility clients
Use Qdrant-style Python and TypeScript clients during Qdrant-to-LambdaDB application migrations.
## MCP
If you want to use LambdaDB from an MCP client such as Claude Desktop, see the MCP guide:
## Client lifecycle
Python: `LambdaDB` supports context manager usage. `__enter__` returns the client, and `__exit__` calls `client.close()` (closing the SDK-owned HTTP client) and makes the client unusable after the `with` block. If you don't use `with`, call `client.close()` when you're done. If you pass a custom `client=`/`async_client=`, you own closing it.
JavaScript/TypeScript: the SDK uses the platform `fetch` API and does not expose a `close()` method. In most environments there's nothing to close explicitly. To cancel requests, pass an `AbortSignal` (for example via request `fetchOptions.signal`). If you provide your own `fetch`/HTTP implementation to the SDK, you own its lifecycle.
Go: the SDK uses `net/http` and does not require an explicit `Close()` in typical usage. Responses are closed internally. If you provide a custom `http.Client`/`Transport`, you own its lifecycle (for example, managing idle connections) and should shut it down according to your application's needs.