Query embedding processor

Introduced 0.11.0

The query_embedding processor is a search request processor that embeds query text (or images) at search time and injects the resulting vector into the search query. This enables natural language search without requiring the client to call an embedding API separately.

The processor reads query parameters from the ext.query_embedding section of the search request and rewrites the query to include the computed vector.

Syntax

PUT /_search/pipeline/my-search-pipeline
{
  "request_processors": [
    {
      "query_embedding": {
        "model_id": "amazon.titan-embed-text-v2:0",
        "provider": "bedrock",
        "dimensions": 1024,
        "mode": "template",
        "query_template": "{\"query\":{\"nested\":{\"path\":\"chunks\",\"query\":{\"knn\":{\"chunks.embedding\":{\"vector\":${embedding},\"k\":10}}},\"inner_hits\":{\"_source\":[\"chunks.text\"]}}}}",
        "provider_config": {
          "region": "us-east-2"
        }
      }
    }
  ]
}

How it works

Client sends:
┌──────────────────────────────────────────────┐
│ GET /documents/_search?search_pipeline=...   │
│ {                                            │
│   "ext": {                                   │
│     "query_embedding": {                     │
│       "text": "environmental impact"         │
│     }                                        │
│   }                                          │
│ }                                            │
└──────────────────────────┬───────────────────┘
                           │
                  query_embedding processor
                           │
          1. Extract "environmental impact"
          2. Call Bedrock Titan Text v2
          3. Get 1024-dim vector
          4. Inject into query template
                           │
                           ▼
Rewritten query:
┌──────────────────────────────────────────────┐
│ {                                            │
│   "query": {                                 │
│     "nested": {                              │
│       "path": "chunks",                      │
│       "query": {                             │
│         "knn": {                             │
│           "chunks.embedding": {              │
│             "vector": [0.012, -0.045, ...],  │
│             "k": 10                          │
│           }                                  │
│         }                                    │
│       }                                      │
│     }                                        │
│   }                                          │
│ }                                            │
└──────────────────────────────────────────────┘

Request fields

Field	Data type	Required/Optional	Description
model_id	String	Required	Embedding model identifier.
provider	String	Required	Embedding provider: bedrock, openai, or http.
dimensions	Integer	Optional	Embedding dimensions. Default is 1536.
mode	String	Optional	Query rewrite mode: template or path. Default is template.
query_template	String	Conditional	JSON template with ${embedding} and ${text} placeholders. Required when mode is template.
embedding_path	String	Conditional	Dot-notation path to inject the vector (e.g., query.knn.vector). Required when mode is path.
on_failure_action	String	Optional	fail (reject query) or passthrough (return query unchanged). Default is fail.
provider_config	Object	Optional	Provider-specific configuration.
reference_config	Object	Optional	Configuration for URI resolution when using image_uri.
tag	String	Optional	The processor's identifier.
description	String	Optional	A description of the processor.

Query input modes

The ext.query_embedding object in the search request supports three input modes:

Text-only

{
  "ext": {
    "query_embedding": {
      "text": "environmental impact near the river"
    }
  }
}

Image-only

{
  "ext": {
    "query_embedding": {
      "image_uri": "s3://bucket/query-image.jpg"
    }
  }
}

Or with inline base64:

{
  "ext": {
    "query_embedding": {
      "image_data": "<base64-encoded image>",
      "image_mime_type": "image/jpeg"
    }
  }
}

Multimodal (text + image)

{
  "ext": {
    "query_embedding": {
      "text": "satellite image of urban area",
      "image_data": "<base64-encoded image>",
      "image_mime_type": "image/jpeg"
    }
  }
}

Rewrite modes

Template mode (default)

The processor replaces ${embedding} with the JSON-encoded vector array and ${text} with the escaped query text:

{
  "query_embedding": {
    "mode": "template",
    "query_template": "{\"query\":{\"knn\":{\"my_field\":{\"vector\":${embedding},\"k\":5}}}}"
  }
}

Path mode

The processor injects the vector at a dot-notation path in the original query:

{
  "query_embedding": {
    "mode": "path",
    "embedding_path": "query.knn.chunks.embedding.vector"
  }
}

Using the processor

Example 1: Semantic search pipeline

PUT /_search/pipeline/semantic-search
{
  "request_processors": [
    {
      "query_embedding": {
        "model_id": "amazon.titan-embed-text-v2:0",
        "provider": "bedrock",
        "dimensions": 1024,
        "mode": "template",
        "query_template": "{\"query\":{\"nested\":{\"path\":\"chunks\",\"query\":{\"knn\":{\"chunks.embedding\":{\"vector\":${embedding},\"k\":10}}},\"inner_hits\":{\"_source\":[\"chunks.text\"],\"size\":3}}}}",
        "provider_config": {
          "region": "us-east-2"
        }
      }
    }
  ]
}

Search using natural language:

GET /documents/_search?search_pipeline=semantic-search
{
  "ext": {
    "query_embedding": {
      "text": "What are the environmental regulations for river discharge?"
    }
  }
}

Example 2: Multimodal image search

PUT /_search/pipeline/geo-search
{
  "request_processors": [
    {
      "query_embedding": {
        "model_id": "amazon.titan-embed-image-v1",
        "provider": "bedrock",
        "dimensions": 1024,
        "mode": "template",
        "query_template": "{\"query\":{\"nested\":{\"path\":\"chunks\",\"query\":{\"knn\":{\"chunks.embedding\":{\"vector\":${embedding},\"k\":10}}},\"inner_hits\":{\"_source\":[\"chunks.tile_id\",\"chunks.bbox\"],\"size\":5}}}}",
        "provider_config": {
          "region": "us-east-1"
        }
      }
    }
  ]
}

Search with text-to-image:

GET /geospatial-imagery/_search?search_pipeline=geo-search
{
  "ext": {
    "query_embedding": {
      "text": "high concentration nitrogen dioxide pollution"
    }
  }
}

Example 3: Hybrid search pipeline (BM25 + kNN)

Combine keyword and semantic search with score normalization:

PUT /_search/pipeline/hybrid-search
{
  "request_processors": [
    {
      "query_embedding": {
        "model_id": "amazon.titan-embed-text-v2:0",
        "provider": "bedrock",
        "dimensions": 1024,
        "mode": "template",
        "query_template": "{\"query\":{\"hybrid\":{\"queries\":[{\"match\":{\"title\":{\"query\":\"${text}\"}}},{\"nested\":{\"path\":\"chunks\",\"query\":{\"knn\":{\"chunks.embedding\":{\"vector\":${embedding},\"k\":10}}}}}]}}}",
        "provider_config": {
          "region": "us-east-2"
        }
      }
    }
  ],
  "phase_results_processors": [
    {
      "normalization-processor": {
        "normalization": { "technique": "min_max" },
        "combination": {
          "technique": "arithmetic_mean",
          "parameters": { "weights": [0.3, 0.7] }
        }
      }
    }
  ]
}