XRAPH/Research/Technical note

Content-Addressed Node Identity for Multi-Source Knowledge Graphs

Deriving node identity from a normalised natural key rather than allocating it makes multi-source ingestion idempotent, order independent and convergent. Analyses the two places it fails, and argues the error asymmetry requires strictness by default.

Type
Technical note
Year
2023
Status
Draft
Length
3 min read
Focus area
Agent Knowledge Distillation & Graphing

#Abstract

Knowledge graphs assembled from several systems of record face an identity problem before they face any other. This note describes deriving node identity as a cryptographic hash of a normalised natural key rather than allocating it at insert time, states the three properties that follow, and analyses two failure modes in detail, including one for which no satisfactory solution is offered.

#The multi-source problem

An industrial site holds the same physical asset in a maintenance system, a process historian, engineering drawings and at least one spreadsheet. Each carries a different identifier, identifier hygiene across them is poor, and at least one contains a long-standing transcription error that downstream reports now depend on.

With allocated identity, ingesting a second source produces a duplicate node, requiring entity resolution, a merge operation, an inverse for incorrect merges, and a policy for edges attached to a merged node. Each of these is a permanent source of operational load rather than a one-off cost.

#Construction

id(n)  =  H ⁣(tκν(x))\mathrm{id}(n) \;=\; H\!\left(t \,\|\, \kappa \,\|\, \nu(x)\right)
(1)
Identity derived from the normalised tuple

where t is the tenant, κ the node kind, x the source identifier and ν a site-specific normalisation function. H is a cryptographic hash, so collision is not an operational concern .

#Three properties

Idempotent ingestion. Re-running an import writes identical identifiers with identical content. There is no lookup before insert and no constraint handling, so a failed import is safe to re-run. This removes a category of recovery decision rather than automating it .

Convergence. Two sources whose normalisations agree land on the same node without either knowing the other exists. Attributes accumulate instead of forking.

Order independence. An edge may reference a node that does not yet exist, because the identifier is computable. Import parallelism therefore requires no coordination.

id(n1)=id(n2)    ν(x1)=ν(x2)\mathrm{id}(n_1) = \mathrm{id}(n_2) \iff \nu(x_1) = \nu(x_2)
(2)
Convergence holds exactly when normalisations agree

This equivalence is the whole result, and its consequence is that the entire difficulty is relocated into ν. That is a real gain, because ν is a pure function with no dependencies and is therefore testable, whereas an entity resolution pipeline is not.

#Failure mode one: normalisation ambiguity

Whether a family of similar identifiers denotes one asset is site specific and cannot be decided by the library. The two errors are not symmetric.

  • Too strict produces two nodes for one asset. This is visible: an operator observes duplicates and reports them.
  • Too permissive produces one node for two assets. This is silent: attributes merge, nothing appears broken, and traversals return confidently incorrect neighbourhoods.
The asymmetry argues for strictness by default. A visible duplicate is a support ticket. A silent merge is a wrong answer nobody knows to question.

Because ν is pure, a site's rules are expressible as a table of inputs and expected outputs, so an operator's assertion that two identifiers denote one asset becomes a test case rather than a schema change.

#Failure mode two: temporal reuse

A source system may reuse an identifier across time, where a tag denoted one asset until a replacement and a different one afterwards. Content-addressed identity has no temporal component, so both map to one node.

The implemented handling is a supersedes edge with the resolver preferring the most recent node. This is a workaround: it requires the reuse to have been noticed and recorded, which is precisely the information the source system failed to encode.

A principled construction would include a validity interval in the hashed tuple, making identity a function of tenant, kind, identifier and epoch. This is not implemented, and the obstacle is that the epoch is unknown at ingest time for exactly the sources that require it. No solution is offered here.

#Interaction with a rebuildable write path

Where every node write is a command producing a versioned event and the property graph is a derived projection, deterministic identity and replayable ingestion compose. A graph rebuilt from the event log is identical rather than merely equivalent, which is verifiable by comparison .

#Why a graph rather than a hierarchy

Equipment hierarchies from industrial standards are trees and plants are not: shared headers, site utilities and shared spares each violate the tree. Retaining the hierarchy as one edge type within a multi-relational graph preserves both, which is the representational argument the knowledge graph literature makes generally .

#Limitations

No comparison against a conventional match-and-merge pipeline on the same source data is reported. That comparison, measuring duplicate rate and incorrect merge rate under both approaches, is the experiment that would establish the practical claim, and it has not been run.

Cross-tenant reference data is duplicated per tenant, which is correct for isolation and wasteful for genuinely shared catalogues. Changing ν after ingestion re-keys every affected node with no migration path short of full re-import.

References

  1. [1]Ralph C. Merkle, A Digital Signature Based on a Conventional Encryption Function, Advances in Cryptology (CRYPTO 87), Springer, 1988doi:10.1007/3-540-48184-2_32
  2. [2]Aidan Hogan et al., Knowledge Graphs, ACM Computing Surveys, vol. 54, no. 4, 2021doi:10.1145/3447772
  3. [3]Shaoxiong Ji, Shirui Pan, Erik Cambria, Pekka Marttinen, Philip S. Yu, A Survey on Knowledge Graphs: Representation, Acquisition, and Applications, IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 2, pp. 494-514, 2022doi:10.1109/TNNLS.2021.3070843
  4. [4]International Electrotechnical Commission, Enterprise-Control System Integration, IEC 62264 (ISA-95)
  5. [5]Martin Kleppmann, Designing Data-Intensive Applications, O'Reilly Media, 2017
  6. [6]Pat Helland, Life beyond Distributed Transactions: An Apostate's Opinion, Conference on Innovative Data Systems Research (CIDR), 2007