Skip to main content
Version: 0.13.0

Importing a vocabulary

Introduced 0.13.0

Concept expansion shows how to index a vocabulary: each edge is a document in a graph_edge field naming a source and a target. That page assumes you already have those edges. This page is about where they come from — how a published vocabulary, or a road network, becomes concepts and relations without your having to hand-write them.

note

This is a library-level capability, not a REST endpoint. There is no "upload your SKOS file" API. The readers described here produce concepts and relations that are then indexed as ordinary graph_edge documents, exactly as shown in Concept expansion. The rules below still matter to you, because they determine what concept identifiers your documents must carry.

Two rules that decide whether re-importing is safe​

Identity comes from the IRI, and is never assigned​

A concept's numeric identity is derived from its IRI by hashing it. It is never handed out in read order.

This is the single most important property of the import. If concepts were numbered sequentially as they were read, the numbering would depend on the order of the file — so re-importing an updated vocabulary would renumber it, and every document already linked to a concept would silently point at a different concept. Nothing would error. Your searches would just quietly become wrong.

Because the identity is a pure function of the IRI, the same concept gets the same identity on every import, from any file, in any order.

If two distinct IRIs ever did produce the same identity, the import fails and names them, rather than merging two concepts into one.

Inverse predicates are turned around, not stored twice​

SKOS lets an author write a hierarchy in either direction: broader pointing up, or narrower pointing down. Both are legal, and real vocabularies use both — sometimes in the same file.

The importer normalizes them. skos:narrower is turned around and stored as broader, so the edge always points from the specific to the general.

This is not tidiness. A traversal that had to know which spelling an author happened to prefer would return half an answer without saying so — you would ask for everything beneath a concept and get only the part written in the direction the traversal expected.

RDF-shaped vocabularies​

SKOS, OWL and RDFS differ only in which predicates mean what, so they share one reader and are described by a vocabulary profile.

ProfilePredicates read
SKOSskos:broader and skos:broaderTransitive as broader; skos:narrower and skos:narrowerTransitive turned around into broader; skos:prefLabel as the preferred label; skos:altLabel and skos:hiddenLabel as alternates
OWLrdfs:subClassOf as subclass_of; rdfs:label as the preferred label; skos:altLabel as alternates
DEFAULTBoth of the above, for files that mix them

The relation names on the right — broader, subclass_of — become the graph_edge field names you traverse, which is why graph_traversal takes subclass_of as its field.

Serializations read: Turtle and N-Triples.

What the parser refuses​

The parser throws rather than skipping when it meets any of these:

  • Blank nodes. Importing around one would silently drop what it says.
  • Collections and anonymous nodes. Same reason.
  • @base. Use absolute IRIs.

This is deliberate. A vocabulary importer that skips what it does not understand produces a graph that is quietly missing relations, and a traversal over it returns a confident, incomplete answer. Failing the import is recoverable; a silently partial graph is not.

Domain graphs: road networks, social graphs, citation graphs​

Not every graph worth indexing is a published vocabulary. A road network, a follower graph and a citation graph share a shape with a taxonomy: an edge is two identifiers plus some facts about the pair. They are read from an edge list and arrive at the index the same way, so nothing downstream has to know which kind of file it was given.

Numeric identifiers are kept exactly​

An OpenStreetMap node already has an ID, and it is the ID every other system uses to talk about that junction. Hashing it would make the index unable to join to anything — including the next export of the same data. So a value that is already a number is used as it stands.

Identifiers that are not numbers — a username, a DOI — are hashed exactly as a concept IRI is, so they are stable across imports for the same reason.

warning

The two must not be mixed, and a mixed file is refused. If some rows had numeric identifiers and others textual ones, a single junction could silently become two nodes — one under its own ID, one under the hash of that ID written as text — and the graph would then be missing every edge between the two halves.

Weights ride along on the edge​

Any column beyond the first two becomes an attribute of the edge. That is where the graph design puts weights: on the edge's own document, so a road's length or speed limit can be updated without rewriting the topology.

Formats not yet read directly​

OBO stanza files and the WordNet database files are not triples, so they do not share the RDF reader. They are designed for — and will emit the same concepts and relations, so nothing downstream changes when they arrive.

An OpenStreetMap extract exported as an edge list imports today; reading .osm.pbf natively does not, as that carries a Protocol Buffers dependency decision that has not been made.

Next​