Retrieval-augmented generation processor
The retrieval_augmented_generation processor is a search results processor that you can use in conversational search for retrieval-augmented generation (RAG). The processor intercepts query results, retrieves previous messages from the conversation from the conversational memory, and sends a prompt to a large language model (LLM). After the processor receives a response from the LLM, it saves the response in conversational memory and returns both the original Lucenia query results and the LLM response.
The retrieval_augmented_generation processor supports only OpenAI and Amazon Bedrock models.
Request fields
The following table lists all available request fields.
| Field | Data type | Description |
|---|---|---|
model_id | String | The ID of the model used in the pipeline. Required. |
context_field_list | Array | A list of fields contained in document sources that the pipeline uses as context for RAG. Required. For more information, see Context field list. |
system_prompt | String | The system prompt that is sent to the LLM to adjust its behavior, such as its response tone. Can be a persona description or a set of instructions. Optional. |
user_instructions | String | Human-generated instructions sent to the LLM to guide it in producing results. |
tag | String | The processor’s identifier. Optional. |
description | String | A description of the processor. Optional. |
Context field list
The context_field_list is a list of fields contained in document sources that the pipeline uses as context for RAG. For example, suppose your Lucenia index contains a collection of documents, each including a title and text:
{
"_index": "qa_demo",
"_id": "SimKcIoBOVKVCYpk1IL-",
"_source": {
"title": "Abraham Lincoln 2",
"text": "Abraham Lincoln was born on February 12, 1809, the second child of Thomas Lincoln and Nancy Hanks Lincoln, in a log cabin on Sinking Spring Farm near Hodgenville, Kentucky.[2] He was a descendant of Samuel Lincoln, an Englishman who migrated from Hingham, Norfolk, to its namesake, Hingham, Massachusetts, in 1638. The family then migrated west, passing through New Jersey, Pennsylvania, and Virginia.[3] Lincoln was also a descendant of the Harrison family of Virginia; his paternal grandfather and namesake, Captain Abraham Lincoln and wife Bathsheba (née Herring) moved the family from Virginia to Jefferson County, Kentucky.[b] The captain was killed in an Indian raid in 1786.[5] His children, including eight-year-old Thomas, Abraham's father, witnessed the attack.[6][c] Thomas then worked at odd jobs in Kentucky and Tennessee before the family settled in Hardin County, Kentucky, in the early 1800s.[6]\n"
}
}
You can specify that only the text contents should be sent to the LLM by setting "context_field_list": ["text"] in the processor.
Example
The following example demonstrates using a search pipeline with a retrieval_augmented_generation processor.
Creating a search pipeline
The following request creates a search pipeline containing a retrieval_augmented_generation processor for an OpenAI model:
PUT /_search/pipeline/rag_pipeline
{
"response_processors": [
{
"retrieval_augmented_generation": {
"tag": "openai_pipeline_demo",
"description": "Demo pipeline Using OpenAI Connector",
"model_id": "gnDIbI0BfUsSoeNT_jAw",
"context_field_list": ["text"],
"system_prompt": "You are a helpful assistant",
"user_instructions": "Generate a concise and informative answer in less than 100 words for the given question"
}
}
]
}
Using a search pipeline
Combine an Lucenia query with an ext object that stores generative question answering parameters for the LLM:
GET /my_rag_test_data/_search?search_pipeline=rag_pipeline
{
"query": {
"match": {
"text": "Abraham Lincoln"
}
},
"ext": {
"generative_qa_parameters": {
"llm_model": "gpt-3.5-turbo",
"llm_question": "Was Abraham Lincoln a good politician",
"memory_id": "iXC4bI0BfUsSoeNTjS30",
"context_size": 5,
"message_size": 5,
"timeout": 15
}
}
}