HTTP inference provider

Introduced 0.11.0

Several processors call an InferenceProvider for structured model output (text, scores, boxes, masks) rather than embedding vectors:

• ocr
• multimodal_rerank
• image_segment (same HTTP regions response; the processor is documented in later versions)

This page is the http inference contract: the JSON Lucenia POSTs and the JSON it parses. Point provider_config.endpoint at a service that speaks this schema. Wrap any other API with an adapter that emits this JSON.

Configure it

{
  "ocr": {
    "model_id": "paddle-ocr-v4",
    "provider": "http",
    "provider_config": {
      "endpoint": "http://ocr.internal:8080/infer",
      "response_text_path": "$.outputs",
      "auth_header": "Bearer <token>"
    }
  }
}

Key	Required	Default	Description
endpoint	Yes	—	POST URL. Missing endpoint fails pipeline creation.
response_text_path	No	$.outputs	JSONPath to an array of output objects, one per input, same order.
auth_header	No	—	Value of the Authorization header.

Put endpoint under provider_config. HTTP 429 and 5xx are treated as retryable; other non-200 responses fail the call.

Request Lucenia sends

POST application/json:

{
  "inputs": [
    {
      "image": "<base64>",
      "image_mime_type": "image/png",
      "prompt": "Extract all visible text..."
    }
  ],
  "task_type": "ocr",
  "model": "paddle-ocr-v4"
}

Field	When present
inputs[].text	Text-only or multimodal inputs (rerank, summarize).
inputs[].image	Standard base64 of the image bytes. No data-URL prefix.
inputs[].image_mime_type	MIME of that image (image/png, image/jpeg, …).
inputs[].prompt	Custom instruction from the processor prompt parameter.
task_type	ocr, caption, rerank, summarize, classify, extract_layout, or image_segment.
model	The processor's model_id. Omitted if empty.

The ocr processor sends inputs of length 1. Other callers may send more than one.

Response Lucenia parses

Default path $.outputs. Length must equal inputs.length or the call fails.

{
  "outputs": [
    {
      "text": "extracted text...",
      "confidence": 0.95,
      "boxes": [
        {
          "text": "extracted",
          "x": 0.05,
          "y": 0.10,
          "width": 0.30,
          "height": 0.08,
          "confidence": 0.98
        }
      ]
    }
  ]
}

Field	Type	Used by
text	String	ocr writes this onto the block (or target_text_field in reference mode).
confidence	Number	ocr → ocr_confidence.
boxes	Array	ocr → ocr_bounding_boxes. Coordinates as Lucenia stored them: x, y, width, height plus text and confidence.
regions	Array	image_segment. If regions is non-empty, Lucenia treats the output as segmentation and ignores the OCR fields.

A regions element:

{
  "label": "building",
  "score": 0.91,
  "polygon": [[0.1, 0.2], [0.4, 0.2], [0.4, 0.5], [0.1, 0.5]],
  "x": 0.1,
  "y": 0.2,
  "width": 0.3,
  "height": 0.3,
  "area_fraction": 0.09
}

polygon is [[x, y], ...] in unit image space [0, 1]. Axis-aligned x / y / width / height and area_fraction are used when present; otherwise they are derived from the polygon.

response_text_path can point at a different array ($.result.items). If the upstream API says transcription instead of text, the adapter must map it.

Request and response vs embedding HTTP and Bedrock

Caller	Request	Response
Inference http (this page)	{ "inputs": [...], "task_type": "...", "model": "..." }	{ "outputs": [{ "text", "confidence", "boxes" }] }
Embedding http	Text-only: { "texts": [...], "model": "..." }. Any image in the batch: { "inputs": [{ "text" / "image" / both }], "model": "..." }	{ "embeddings": [[...], ...] }
Inference bedrock	Claude Messages: anthropic_version, system, messages[].content with { "type": "image", "source": { "type": "base64", "data": "..." } }	{ "content": [{ "type": "text", "text": "..." }] }

Adapter sketch

A typical self-hosted OCR setup:

1. Run PaddleOCR / Tesseract / a VLM behind whatever API it already has.
2. Put a small HTTP service at provider_config.endpoint.
3. That service reads inputs[0].image (base64), decodes it, calls the engine, and returns { "outputs": [{ "text": "...", "confidence": 0.9 }] }.
4. Set the ocr processor provider to http and provider_config.endpoint to that service.

Task types

task_type	Typical input	Typical output fields
ocr	Image	text, confidence, boxes
caption	Image	text
rerank	Text (Query:\n...\nPassage:\n...)	confidence as the score
summarize	Text	text
classify	Text or image	text / confidence
extract_layout	Image	text / boxes
image_segment	Image	regions

Built-in processors pass these strings. A custom InferenceProvider SPI can use others; the HTTP provider copies task_type into the JSON.