Blocking
In record matching, sorting records into groups that share a cheap key, such as the same postal code, so that only records inside one group are compared with each other.
Blocking exists because comparing every record with every other record does not scale. Its cost is that two matching records placed in different blocks are never compared, so a typo in the blocking key hides a duplicate.
Where it comes up
- Near-duplicate recordsTwo rows describe the same real thing but differ enough that an exact comparison misses them.
- The same entity in two systemsOne person or organization exists in two systems under different identifiers, with nothing linking the records.
See also
- Record linkageFinding the records in two or more datasets that refer to the same person, company, or thing, when the datasets share no common identifier.
- Entity resolutionDeciding which records refer to the same real-world person, company, or thing, when no shared identifier says so.
- Match thresholdThe similarity score at or above which two records are treated as the same thing. Pairs scoring below it are left as separate records.