Query data

A search query, or query, is a request for information about data in LambdaDB collections.

LambdaDB supports several search methods:

• Search for exact values: search for exact values or ranges of numbers, dates, IPs, or strings.
• Full-text search: use full text queries to query unstructured textual data and find documents that best match query terms.
• Vector search: store vectors in LambdaDB and use approximate nearest neighbor (ANN) to find vectors that are similar, supporting use cases like semantic search.

Common parameters for the request body of a query are as follows:

Query results are returned in the following format:

Example query request to LambdaDB is as follows:

url = f"https://{baseUrl}/projects/{projectName}/collections/{collectionName}/query"
headers = {"content-type" : "application/json", "x-api-key" : f"{YOUR_API_KEY}"}

query = {
    "size": 10,
    "query": { }
}

r = requests.post(url, headers=headers, json=query)

The response will look like this:

{
  "took": 76,
  "maxScore": 10.691499,
  "total": 1,
  "docs": [
    {
      "collection": "example_collection",
      "score": 10.691499,
      "doc": {
        "id": "33201222",
        "url": "https://en.wikipedia.org/wiki/LambdaDB",
        "title": "LambdaDB is awesome",
        "text": null
      }
    }
  ]
}

Matched documents are ordered by similarity from most similar to least similar by default. Similarity is expressed as a score, and it is calculated based on BM25 algorithm for full-text search and a configured similarity metric for vector search.

​

Query string query

LambdaDB supports a simple query string query that uses the Lucene Query Syntax to search documents. Specifically, term, phrase, wildcard, boolean operators, range operators, and special character escaping are supported.

A query to a collection with object field type requires special handling.

An example index configurations:

{
  "metadata": {
    "type": "object",
    "objectMapping": {
      "url": {
        "type": "keyword"
      },
      "author": {
        "type": "keyword"
      }
    }
  }
}

An example query:

{
  "queryString": {
    "query": "metadata.author:Alan"
  }
}

​

Vector query

A k-nearest neighbor (kNN) search finds the k nearest vectors to a query vector, as measured by a similarity metric.

An example vector query:

{
  "knn": {
    "field": "example_vector_field",
    "queryVector": [
        0.030255454,
        -0.058824085,
        -0.065448694,
        -0.03987034,
        0.060786933,
        -0.15469691,
        -0.043918714,
        0.057719983,
        0.054530356,
        0.007080819
    ],
    "k": 5,
    "numCandidates": 20
  }
}

An example vector query with a filter for better performance:

{
  "knn": {
    "filter" : {
      "queryString": {
          "query": "node_type:NODE AND \"https://example.com/books/5514276\"",
          "defaultField": "metadata.url"
      }
    },
    "field": "example_vector_field",
    "queryVector": [
        0.030255454,
        -0.058824085,
        -0.065448694,
        -0.03987034,
        0.060786933,
        -0.15469691,
        -0.043918714,
        0.057719983,
        0.054530356,
        0.007080819
    ],
    "k": 5,
    "numCandidates": 20
  }
}

​

Boolean query

A query that matches documents matching boolean combinations of other queries. The boolean query maps to Lucene BooleanQuery. It is built using one or more boolean clauses, each clause with a typed occurrence. The occurrence types are:

Below is an example of a boolean query:

{
  "bool": [
    {
      "queryString": {
        "query": "node_type:NODE"
      },
      "occur" : "filter"
    },
    {
      "queryString": {
        "query": "content:LambdaDB"
      },
      "occur" : "should"
    }
  ]
}

This query will return top documents that are both of type NODE and have the highest score value.

​

Boost query

Boost values that are less than one will give less importance to this query compared to other ones while values that are greater than one will give more importance to the scores returned by this query.

{
  "bool": [
    {
      "queryString": {
        "query": "node_type:NODE"
      },
      "boost": 0.8
    },
    {
      "queryString": {
        "query": "content:LambdaDB"
      },
      "boost": 0.2
    }
  ]
}

The final score of the matched document is calculated as proportional to the boost values.

​

Hybrid query

You can mix the aforementioned queries with a vector query to get better relavance.

An example hybrid query:

{
  "bool": [
    {
      "queryString": {
          "query": "content:LambdaDB"
      },
      "occur": "should",
      "boost": 0.7
    },
    {
      "knn": {
        "filter" : {
         "queryString": {
            "query": "\"https://lambdadb.ai\"",
            "defaultField": "metadata.url"
          }
        },
        "field": "text_embedding",
        "queryVector": [ ... {float array} ... ],
        "k": 5,
        "numCandidates": 20
      },
      "occur": "should",
      "boost": 0.3
    }
  ]
}

To explain the above example, it works as follows:

A total of two queries are combined into a bool query, performing a hybrid query that combines a queryString and a knn.

The score of documents matching in the queryString is boosted by 0.7 during the calculation.

In the knn, a pre-filtering is performed using a filter, followed by a vector query that calculates the scores of the top 5 most similar documents with a boost of 0.3.

​

Sort search results

LambdaDB allows you to add one or more sorts on specific fields. Each sort can be reversed as well. The sort is defined on a per field level.

The sort order option can have the following values:

• asc: sorts in ascending order
• desc: sorts in descending order

Assuming the following index configurations:

{
  "indexConfigs": {
    "post_date": {
      "type": "datetime"
    },
    "user": {
      "type": "keyword"
    },
    "name": {
      "type": "keyword"
    },
    "age": {
      "type": "integer"
    }
  }
}

You can sort search results as follows:

{
  "query" : {
    "queryString" : { "query" : "name:LambdaDB" }
  },
  "sort" : [
    { "post_date" : "desc" },
    { "age" : "desc" }
  ]
}

When sorting on a field, scores are not computed. By setting trackScores to true, scores will still be computed and tracked.

{
  "query" : {
    "queryString" : { "query" : "name:LambdaDB" }
  },
  "sort" : [
    { "name" : "asc" },
    { "age" : "desc" }
  ],
  "trackScores": true
}

​

Query limits

The query result size is affected by the dimension of the vectors and whether vector values are included in the result.