Rounding and precision loss

• A spreadsheet showed a rounded value, and the displayed value was what got copied or exported.
• A very long number was stored as a number instead of text, and its final digits were replaced or folded into scientific notation.
• Values passed through a system that keeps fewer decimal places than the one before it.

• Line items no longer add up to their totals, by small amounts that are tedious to trace.
• Long identifiers stop matching their originals, so joins fail on exactly the rows that were altered.
• Small rounding differences grow when values are multiplied by large quantities.

• Look for scientific notation in columns that should hold identifiers.
• Check whether long numbers end in a run of zeros more often than chance would allow.
• Recompute a total from its parts and compare. A consistent small gap points to rounding upstream.

• Go back to the source and export again. Lost digits cannot be recovered from the rounded value.
• Treat identifiers as text from the moment they are read, so nothing tries to do arithmetic on them.
• Carry full precision through every step and round once, at the point of presentation.
• Where the source is gone, keep the value as it is and record that its precision is uncertain.

• Numbers stored as textA column of quantities contains symbols, separators, or words that stop software reading it as numbers.
• Identifiers that lost their leading zerosA code that starts with zero, such as a postal code or account number, was read as a number and the zeros were dropped.
• Outliers from entry errorsA value sits far outside the rest because of a slipped key, an extra zero, or a misplaced decimal point.

TermsPrecision, Rounding, Floating point, Identifier