Content-Addressed Node Identity for Multi-Source Knowledge Graphs

#Abstract

Knowledge graphs assembled from several systems of record face an identity problem before they face any other. This note describes deriving node identity as a cryptographic hash of a normalised natural key rather than allocating it at insert time, states the three properties that follow, and analyses two failure modes in detail, including one for which no satisfactory solution is offered.

#The multi-source problem

An industrial site holds the same physical asset in a maintenance system, a process historian, engineering drawings and at least one spreadsheet. Each carries a different identifier, identifier hygiene across them is poor, and at least one contains a long-standing transcription error that downstream reports now depend on.

A concrete instance, lightly disguised. One centrifugal pump appeared as P-101A in the maintenance system, PMP101A in the historian tag namespace, P101-A on the piping diagram, and P-101-A (spare fitted 2019) in a spreadsheet maintained by the reliability engineer. All four denote the same machine. A fifth string, P-101B, denotes a different machine sitting a metre away, and differs from the first by one character.

This is the shape of the problem, and the thing to notice is that no amount of string similarity separates the fourth case from the fifth. The four that match are less similar to each other than the two that do not.

#Two ways to assign identity

With allocated identity, the graph mints an opaque identifier when a node is first written. Ingesting a second source produces a second node for the same asset, so the system needs entity resolution to find such pairs, a merge operation to combine them, an inverse for merges made in error, and a policy for what happens to edges attached to a node that has been merged away.

Each of those four is a permanent source of operational load rather than a one-off integration cost. The merge inverse in particular is where the effort concentrates, because an incorrect merge must be undone after downstream systems have already read the merged node, and the undo has to decide which of the accreted attributes belonged to which original.

With derived identity, the identifier is a function of the content that identifies the asset. Two sources describing one asset compute one identifier without either being aware of the other, which means there is never a pair to resolve and never a merge to invert. The cost is that the entire difficulty moves into that function, which is the trade this note argues is worth making.

#Construction

where t is the tenant, κ the node kind, x the source identifier and ν a site-specific normalisation function. H is a cryptographic hash, so collision is not an operational concern [1].

#Three properties

Idempotent ingestion. Re-running an import writes identical identifiers with identical content. There is no lookup before insert and no constraint handling, so a failed import is safe to re-run. This removes a category of recovery decision rather than automating it [2].

Convergence. Two sources whose normalisations agree land on the same node without either knowing the other exists. Attributes accumulate instead of forking.

Order independence. An edge may reference a node that does not yet exist, because the identifier is computable. Import parallelism therefore requires no coordination.

This equivalence is the whole result, and its consequence is that the entire difficulty is relocated into ν. That is a real gain, because ν is a pure function with no dependencies and is therefore testable, whereas an entity resolution pipeline is not.

#Designing the normalisation function

Since everything now depends on ν, it deserves more than a symbol. What ν does in practice is a sequence of rewrites, and the ordering of those rewrites is itself a design decision.

#What the rewrites actually are

Case folding and whitespace collapse are universal and uninteresting. Beyond those, the rules are site specific, and the ones that mattered on the site above were: strip a set of known separator characters so that P-101A, P101A and P101-A coincide; drop parenthetical annotations, which is what removes the reliability engineer's fitted-spare note; expand a site-local abbreviation table so that PMP and P coincide for the pump kind but not for other kinds; and refuse to alter the trailing unit letter, which is what keeps A and B apart.

1func Normalise(kind Kind, raw string) (string, error) {
2	s := strings.ToUpper(strings.TrimSpace(raw))
3	s = dropParenthetical(s)                  // "P-101-A (spare fitted 2019)" -> "P-101-A"
4	s = stripSeparators(s, siteSeparators)    // "P-101-A" -> "P101A"
5	s, err := expandPrefix(kind, s)           // "PMP101A" -> "P101A", kind-scoped
6	if err != nil {
7		return "", err                        // unknown prefix: fail loudly, do not guess
8	}
9	return s, nil                             // trailing unit letter is never touched
10}
Copy

The error return is the part worth defending. An identifier whose prefix is not in the site's table is not normalised on a best-effort basis; the import fails and names the offending value. A function that silently passes through what it does not understand will produce a stable identifier that happens to be wrong, and stable wrong identifiers are the worst outcome available here, because they accumulate edges.

#Why purity is the property that pays

ν takes a string and a kind and returns a string. It reads no database, calls no service and depends on no previously ingested data. Three consequences follow.

A site's identity rules become a table of inputs and expected outputs, so an operator's assertion that two identifiers denote one asset is expressible as a test case rather than as a schema change or a data migration. The five strings above are a five-line test that any engineer can read and any operator can check.

The rules can be changed and the effect predicted, by running the new ν over the existing identifier corpus offline and diffing the resulting partition. This turns what would be a risky migration into a reviewable diff, which is the only reason changing identity rules on a populated graph is contemplated at all.

And the function can be shipped to the edge. Because it depends on nothing, a gateway can compute node identifiers locally and emit edges referencing nodes it has never seen, which is what makes the order independence property below more than a theoretical nicety.

#Failure mode one: normalisation ambiguity

Whether a family of similar identifiers denotes one asset is site specific and cannot be decided by the library. The two errors are not symmetric.

• Too strict produces two nodes for one asset. This is visible: an operator observes duplicates and reports them.
• Too permissive produces one node for two assets. This is silent: attributes merge, nothing appears broken, and traversals return confidently incorrect neighbourhoods.

The asymmetry argues for strictness by default. A visible duplicate is a support ticket. A silent merge is a wrong answer nobody knows to question.

Because ν is pure, a site's rules are expressible as a table of inputs and expected outputs, so an operator's assertion that two identifiers denote one asset becomes a test case rather than a schema change.

#Failure mode two: temporal reuse

A source system may reuse an identifier across time, where a tag denoted one asset until a replacement and a different one afterwards. Content-addressed identity has no temporal component, so both map to one node.

The implemented handling is a supersedes edge with the resolver preferring the most recent node. This is a workaround: it requires the reuse to have been noticed and recorded, which is precisely the information the source system failed to encode.

A principled construction would include a validity interval in the hashed tuple, making identity a function of tenant, kind, identifier and epoch. This is not implemented, and the obstacle is that the epoch is unknown at ingest time for exactly the sources that require it. No solution is offered here.

It is worth being clear about how bad this is. Temporal reuse is not rare in plants with long lives, and the failure it produces is the silent kind: the maintenance history of two different machines accumulates on one node, and a reliability calculation over that node returns a confident number computed from a fiction. A construction that is strong on the multi-source problem and weak here is not uniformly better than match-and-merge, which at least has somewhere to record that two records are the same asset at different times.

#What this borrows from entity resolution, and what it refuses

Record linkage has a formal foundation going back to the probabilistic framework of Fellegi and Sunter, which treats matching as a decision problem over comparison vectors with explicit error rates [3], and a substantial modern literature on doing it at scale [4] [5]. That work is not in dispute, and where the matching problem is genuinely probabilistic it is the right tool.

The refusal here is narrow. In this setting the sources are internal to one operator, the identifier schemes are documented or discoverable, and somebody on site knows the rules. The decision is therefore deterministic in principle, and encoding it as a probability distribution over string comparisons discards information the operator already has. The P-101A against P-101B case is the argument: a probabilistic matcher must assign those a high similarity and then be corrected, whereas ν is told the trailing letter is significant and gets it right by construction.

The trade is that deterministic identity fails on sources where no rule exists, for example free-text equipment descriptions with no identifier at all. Large public knowledge bases face exactly that and use allocated identity with curation for good reason [6] [7]. The claim here is not that derived identity is generally superior. It is that for multi-source industrial data with real identifier schemes, the deterministic route removes an entire class of operational machinery, and the cases where it fails are enumerable in advance.

#Interaction with a rebuildable write path

Where every node write is a command producing a versioned event and the property graph is a derived projection, deterministic identity and replayable ingestion compose. A graph rebuilt from the event log is identical rather than merely equivalent, which is verifiable by comparison [8].

#Why a graph rather than a hierarchy

Equipment hierarchies from industrial standards are trees [9] and plants are not: shared headers, site utilities and shared spares each violate the tree. Retaining the hierarchy as one edge type within a multi-relational graph preserves both, which is the representational argument the knowledge graph literature makes generally [10] [11].

#Limitations

No comparison against a conventional match-and-merge pipeline on the same source data is reported. That comparison, measuring duplicate rate and incorrect merge rate under both approaches on the same corpus, is the experiment that would establish the practical claim. It has not been run, and until it is, the argument above rests on the structural point that one approach has a merge inverse to maintain and the other does not.

Changing ν after ingestion re-keys every affected node. The offline diff described earlier makes the consequence visible before the change, which is a real mitigation, but the migration itself is a full re-import and there is no incremental path. For a graph of any size this makes identity rules effectively append-only in practice: new rules are added readily, existing ones are changed with reluctance.

Cross-tenant reference data is duplicated per tenant, since the tenant is inside the hashed tuple. This is correct for isolation and wasteful for genuinely shared catalogues such as manufacturer part definitions, where every tenant holds its own copy of the same node. A second identity domain for shared reference data would resolve it and would need its own access rules, which has not been designed.

Finally, the hash is over a normalised tuple and not over the node's attributes, so identity is stable under attribute change by design. This means the construction says nothing about whether two nodes with the same identifier and contradictory attributes represent a data quality problem. Detecting that remains a separate concern, and the graph as built will hold both values without complaint.