Skip to main content
Version: 0.12.0

Memory and content extraction MCP tools

Introduced 0.12.0

Lucenia's built-in MCP server can expose two families of native capabilities as MCP tools, so any MCP client can call them over JSON-RPC without writing a connector:

  • Memory tools — the inference-memory verbs (memory_recall, memory_remember, memory_forget, memory_drift_check, memory_rollup), contributed by the memory module.
  • Content extraction tools — document text/metadata extraction (extract_content, fetch_content), contributed by the ingest-content module.

Both families plug into the MCP server through the same extension point in the ML Commons plugin, so once the MCP server is enabled they are discoverable and callable at the standard /_plugins/_ml/mcp endpoint.

tip

This page covers exposing memory and content extraction as MCP tools. To call the memory verbs directly over their own REST API instead, see the memory verbs under inference memory. To connect Lucenia out to a third-party MCP server, see Connecting to an external MCP server.

Table of contents

Enable the MCP server​

The MCP server is off by default. Turn it on with a single dynamic cluster setting — no node restart is required:

PUT /_cluster/settings
{
  "persistent": {
    "plugins.ml_commons.mcp_server_enabled": "true"
  }
}
SettingTypeDefaultScope
plugins.ml_commons.mcp_server_enabledbooleanfalsenode-scoped, dynamic

While this setting is false, the MCP endpoint returns an error (The MCP server is not enabled...) and no tools are served. Flipping it to true also triggers automatic registration of the memory tools (described below).

How tools are contributed: the MLCommonsExtension SPI​

The ML Commons plugin (ml-common) is an extensible plugin. Other modules contribute MCP tools by implementing the extension interface:

package io.lucenia.ml.common;

public interface MLCommonsExtension {
    List<Tool.Factory<? extends Tool>> getToolFactories();
}

A module declares ml-common as an extended plugin, and at load time ML Commons calls loadExtensions(MLCommonsExtension.class) over the shared classloader, collecting every returned Tool.Factory. Each factory carries a tool type, name, description, and input schema (a JSON Schema string), which is what an MCP client sees in tools/list.

  • The memory module contributes tools through MemoryToolsExtension.
  • The ingest-content module contributes tools through ContentToolsExtension.
note

There are two related-but-distinct registries. Implementing MLCommonsExtension makes a tool type known to the node. A tool only shows up in an MCP client's tools/list after it is written into the cluster-state MCP tool registry (.plugins-ml-mcp-tools), either automatically (memory tools) or by an explicit register call (content tools). See Registering the content tools.

Connect a client​

The built-in server speaks JSON-RPC 2.0 over stateless Streamable HTTP at:

POST /_plugins/_ml/mcp

Only POST is accepted; a GET to this endpoint returns 405 Method Not Allowed. Responses use the JSON-RPC envelope — a JSON-RPC-level error is returned with HTTP 200 OK and an error object, while transport-layer problems (wrong method, disabled server) surface as HTTP status codes.

List the available tools:

curl -X POST "https://localhost:9200/_plugins/_ml/mcp" \
  -H "Authorization: Basic <credentials>" \
  -H "Content-Type: application/json" \
  -d '{ "jsonrpc": "2.0", "id": 1, "method": "tools/list", "params": {} }'

Call a tool with tools/call, passing the tool name and its arguments:

curl -X POST "https://localhost:9200/_plugins/_ml/mcp" \
  -H "Authorization: Basic <credentials>" \
  -H "Content-Type: application/json" \
  -d '{
    "jsonrpc": "2.0",
    "id": 2,
    "method": "tools/call",
    "params": {
      "name": "memory_recall",
      "arguments": { "namespace": "itproj", "user": "tester", "query": "what did we decide about retries?" }
    }
  }'

The result is the standard MCP tool-call shape: a result object carrying content and an isError flag. Each memory and content tool returns its payload as a compact JSON string inside that content.

note

All tool arguments are passed to the underlying tool as strings. Numeric fields (k, max_bytes, older_than_days) and boolean fields (pinned, pinned_exempt, preserve_image_data) are parsed from their string forms, and object-valued fields such as tags are parsed as JSON.

Memory tools​

The memory module exposes five tools. When the MCP server is enabled, they are auto-registered — a cluster component (MemoryMcpToolRegistrar) seeds them into the cluster MCP registry from the elected cluster-manager node once, so they appear in tools/list out of the box. You do not need to register them by hand. Registration re-fires if you enable the server later at runtime, and is idempotent across nodes and restarts.

All five accept an optional tenancy scope made of tenant, namespace, agent, session, and user (each a string); only the dimensions you supply constrain the operation.

Tool nameTypeRequired argumentsPurpose
memory_recallMemoryRecallToolqueryRecall the most relevant memories for a query within a scope, using hybrid lexical+vector retrieval with recency decay. Optional k caps results.
memory_rememberMemoryRememberToolcontentRecord a turn or observation into durable memory. Optional role, fact_type, pinned, tags. Returns the written memory id.
memory_forgetMemoryForgetTool(at least one scope dimension)Delete memories in a scope. At least one scope dimension must be set so forget cannot wipe all memory. Optional pinned_exempt (default true) preserves pinned memories.
memory_drift_checkMemoryDriftTool(at least one of goal, constraints, response)(Re)establish a scope's goal/constraints and/or score a response for topic drift. Returns the drift verdict, score, and corrective texts.
memory_rollupMemoryRollupTool(non-global scope)Consolidate aged long-term memories in a scope into a single summary memory. Optional older_than_days, max_sources.

Selected argument details:

  • memory_remember — fact_type is one of semantic, preference, summary, entity, or anchor; pinned: true exempts the memory from eviction; tags is an arbitrary string key/value JSON object.
  • memory_drift_check — constraints is a comma- or newline-separated list.
  • memory_rollup — a non-global scope is required so a rollup can never sweep every tenant at once.
tip

Recall quality depends on the configured embedding provider. See Inference memory embeddings for how to point memory at a real (semantic or multimodal) embedding model.

Content extraction tools​

The ingest-content module exposes two tools that run Lucenia's document extraction path in memory — no ingest pipeline or document context is required.

Tool nameTypeRequired argumentsPurpose
extract_contentExtractContentToolcontent (base64) or textExtract text and metadata from a supplied payload: PDF, Office documents (docx/xlsx/pptx), HTML, plain text, and images (metadata/EXIF).
fetch_contentFetchContentTooluriFetch a document by URI (s3://, gs://, az://, or https://) using the same reference resolvers as the ingest pipeline, then extract it in one call.

Both return a JSON string carrying a concatenated text field, structured content blocks, and document metadata. Shared optional arguments:

  • mime_type — declared MIME type (for example application/pdf, text/plain, image/png). Required for extract_content when content (base64 bytes) is given; defaults to text/plain when only text is given. For fetch_content, it is detected from the source unless overridden.
  • max_bytes — cap on bytes extracted (and, for fetch_content, fetched). Defaults to 100 MB; fetch_content rejects sources that exceed the cap.
  • preserve_image_data — for images, retain the raw image bytes (base64) in the returned block.

Registering the content tools​

Unlike the memory tools, the content tools are not auto-registered. Their tool types are contributed to the node through ContentToolsExtension, but to make them appear in tools/list you must seed them into the cluster MCP registry once, using the register API:

curl -X POST "https://localhost:9200/_plugins/_ml/mcp/tools/_register" \
  -H "Authorization: Basic <credentials>" \
  -H "Content-Type: application/json" \
  -d '{
    "tools": [
      { "name": "extract_content", "type": "ExtractContentTool" },
      { "name": "fetch_content",   "type": "FetchContentTool" }
    ]
  }'

After registering, extract_content and fetch_content become listable and callable exactly like the memory tools.

Managing registered tools​

The MCP tool registry is managed through these REST APIs (all under /_plugins/_ml/mcp/tools):

Method & pathAction permissionPurpose
POST /_plugins/_ml/mcp/tools/_registercluster:admin/lucenia/ml/mcp_tools/registerAdd tools to the registry.
GET /_plugins/_ml/mcp/tools/_listcluster:admin/lucenia/ml/mcp_tools/listList registered tools.
POST /_plugins/_ml/mcp/tools/_updatecluster:admin/lucenia/ml/mcp_tools/updateUpdate registered tools.
POST /_plugins/_ml/mcp/tools/_removecluster:admin/lucenia/ml/mcp_tools/removeRemove tools from the registry.

Registered tool names must be unique; registering a duplicate name is rejected (the memory auto-registration treats an "already exist" outcome as a no-op).

Security and permissions​

When the security plugin is enabled, reaching the MCP server endpoint (which dispatches tools/list and tools/call) requires the cluster action permission:

cluster:admin/lucenia/ml/mcp/server

Calling a tool then also runs the underlying transport action, so the caller must additionally hold that verb's permission (for example cluster:admin/lucenia/memory/recall for memory_recall). The memory module ships roles that bundle both the memory verbs and MCP-server access:

RoleGrants
memory_readmemory_recall, memory_stats, plus MCP-server access — for retrieval-only agents.
memory_read_writeThe read actions plus remember, forget, drift/check, rollup, plus MCP-server access — the typical agent role.
memory_full_accessEvery memory verb (cluster:admin/lucenia/memory/*) plus MCP-server access.

Managing the registry (_register, _list, _update, _remove) requires the corresponding cluster:admin/lucenia/ml/mcp_tools/* permissions listed above.