Skip to main content
Version: 0.13.0

Concept search tool

Introduced 0.13.0

The ConceptSearchTool, which an agent sees as FindByMeaning, finds every document about a subject — including documents that never use the word.

It resolves a concept to a concept set using the concept expansion, then filters documents on membership of that set. There is no k and no similarity threshold, so within the set the answer is every matching document, not the top ones. That is the difference between this and a vector search: vector search answers what is similar, this answers what is all of it.

And because the honest version of that claim has boundaries, the tool states them in every answer.

Parameters​

ParameterRequiredDefaultDescription
indexYesThe index of documents to search.
vocabularyYesThe index holding the vocabulary.
conceptThe concept identifier to expand from. Give this or term.
termA name to look up in the vocabulary instead of an identifier.
node_fieldNoconceptThe field on your documents carrying concept identifiers — what the concept_link processor wrote.
relationNosubclass_ofThe graph_edge field to expand along.
queryNoA phrase used to rank documents within the expanded set. It does not narrow the set.
sampleNo10How many documents to list, at most.

Register a flow agent that runs the ConceptSearchTool​

POST /_plugins/_agent/agents/_register
{
  "name": "Meaning_Agent",
  "type": "flow",
  "description": "finds every document about a subject",
  "tools": [
    {
      "type": "ConceptSearchTool",
      "name": "DemoConceptSearchTool",
      "parameters": {
        "index": "articles",
        "vocabulary": "mesh",
        "node_field": "concept"
      }
    }
  ]
}

What every answer states​

Which direction the expansion ran​

The answer says the expansion ran upwards to the more general — so documents about broader subjects are included, and documents about narrower ones are not.

This is stated explicitly because a model told only "searched by meaning" assumes both directions. Ask about dog and you get documents about dogs, canines and mammals. You do not get documents about poodles.

warning

This tool does not expand downwards. Asking it about a category will not find documents about that category's subtypes. It says so rather than returning a confidently incomplete list.

The query can walk either way — graph_traversal takes a direction of forward or reverse. FindByMeaning exposes no such parameter and always walks upwards, so if you need subtypes, write the query rather than asking the tool.

How much of the index could ever have been searched​

Every answer reports linked coverage, in the form:

N of M documents in [index] are linked to any concept (P%). Documents that are not linked CANNOT be found this way at all, so this answer is complete over the linked share only.

It reads the linked field that the concept_link processor writes on every document. That field exists for exactly this purpose.

When the coverage figure cannot be obtained, the answer says so — that how much of the index this could ever have searched is unknown. An omitted caveat reads as no caveat.

When the term is not in the vocabulary​

A term the vocabulary does not contain returns nothing was searched for — not an empty result set. An empty result reads as "there is nothing about this", which is a different and false statement.

Concept expansion is a filter, so it composes. In a bool query it intersects with keyword or vector clauses; in a hybrid query the sets are fused. Concept, keyword and vector in one request is supported — see Concept expansion.

Next​