Floating point
The usual way computers store numbers with a fractional part, using a fixed number of binary digits. Most decimal fractions cannot be stored exactly, so 0.1 plus 0.2 comes out as 0.30000000000000004.
The errors are tiny, and they still break exact comparisons and can drift across long sums. Money is usually kept in a decimal type or as a whole number of cents for that reason.
Where it comes up
- Rounding and precision lossDigits were dropped when a value was displayed, exported, or converted, and the file now holds the shortened number.
See also
- PrecisionFor numbers, how finely a value is recorded: 3.14159 carries more precision than 3.1. In matching and error detection the word has a second meaning: of everything that was flagged, how much was flagged correctly.
- RoundingReplacing a number with a nearby one that has fewer digits, such as 2.675 with 2.68.
- IntegerA whole number with no fractional part, such as 42 or -7. It is the natural type for counts.