Docs
Browse Docs
Field guide · Categories and codes

Categories that changed meaning

A label stays the same across the file, but what it covers was redefined partway through.

What it looks like

Before
sale_dateterritoryterritory_definition
2021-06-14West·
2024-06-14West·
2024-06-14Mountain·
After
sale_dateterritoryterritory_definition
2021-06-14Westmap before 2023
2024-06-14Westmap from 2023
2024-06-14Mountainmap from 2023
In this example the West territory was split in 2023 and part of it became Mountain. The label is untouched; the new column records which definition each row was written under.

How it happens

  • A territory, tier, or product line was split, merged, or redrawn, and the old name was reused.
  • A threshold moved, so the same label now applies to a different group.
  • A code was retired and later assigned to something new.

What it breaks

  • Trends show a jump or a drop on the day the definition changed, and it is read as a real change in the business.
  • Comparisons between periods set two different groups against each other under one name.
  • No check catches it, because every value is a valid label.

How to spot it

  • Chart the row count for each category over time. A sudden step with no known cause is a sign.
  • Look for categories that first appear partway through the file, and see which older category shrank at the same moment.
  • Ask the people who maintain the list whether it has been revised, and when.

How to fix it

  • Establish the date of each change from outside the file. The data alone rarely proves it.
  • Record which definition each row was written under, in its own column.
  • Restate old rows under the new definition only where a finer field, such as state or product, makes that possible.
  • Where old rows cannot be restated, say so, and compare periods only within one definition.

TermsSchema drift, Reference data, Consistency, Provenance

Karg checks for many of these when it profiles a file, applies the fixes it is confident about, and sets the rest aside for your review. See how it works, or have our team clean your data for you.