The Trust Pipeline: Identity, Conflict, Provenance, and Schema

Semantica's graph is only as trustworthy as the decisions made before a query reaches it. This chapter follows those decisions from duplicate source records to canonical entities, reconciled values, provenance history, and schema validation. The goal is practical: understand which controls make graph-native context more credible, where human review belongs, and why none of these controls should be confused with proof that a source is true.

A graph can make a bad relationship easier to find. If the same organization appears under four names, its relationships may be split across four nodes. If three sources disagree about a vulnerability score, silently choosing one value hides an important fact about the evidence. If an extracted property has no source or version history, an auditor cannot tell whether it was observed, inferred, corrected, or overwritten. Quality therefore is not a final polish applied after graph construction. It is a sequence of state changes that determines what the graph means.

First decide what counts as the same entity

Deduplication and conflict resolution answer different questions. Deduplication asks whether two records refer to one real-world thing. Conflict resolution asks which assertion should be used when several records have already been recognized as describing that thing. Reversing the order turns aliases into apparent disagreements and makes the resulting graph harder to interpret.

Imagine four feeds describing one threat actor as “APT29,” “APT-29,” “Cozy Bear,” and “Midnight Blizzard.” A name-only comparison catches some of these variations, but not all. The names can be completely different even when an alias field, a stable identifier, or a shared relationship makes the identity clear. Semantica's duplicate detector combines configurable similarity signals and returns candidates with a similarity score, a confidence score, and reasons for the match. It also excludes type mismatches structurally, so lowering a threshold cannot make a person and an organization become duplicates merely because their text is similar.

The detector can compare every pair, form transitive groups, or compare a new batch with an existing canonical set. Pairwise comparison is easy to understand and useful for smaller collections, but its cost grows quadratically with the number of entities. Cluster-oriented methods reduce the number of comparisons for larger sets. Incremental detection avoids comparing a new record with every other historical record when the existing set is already canonicalized. The choice is not an abstract algorithm preference. It depends on whether the system is cleaning a bounded import or keeping up with a stream of new assertions.

Grouping also changes the unit of judgment. If record A matches B and B matches C, a union-find grouping can place all three in one candidate group even when A and C do not pass the direct threshold. That is useful for aliases, but it means a single transitive link can widen the group. Thresholds must therefore be reviewed against false merges as well as missed matches. A low threshold may recover semantically different names; it may also combine entities that share a common property without sharing an identity.

Once a group is accepted, EntityMerger applies a strategy. keepmostcomplete favors the record with the richest attributes and adds missing information from other records. keepfirst is more appropriate when the first source is a designated golden record.