What are the pros and cons of fixed-point arithmetics vs floating-point arithmetics?

From Wikipedia:

In computing, fixed-point is a method of representing fractional (non-integer) numbers by storing a fixed number of digits of their fractional part. ... More generally, the term may refer to representing fractional values as integer multiples of some fixed small unit, e.g. a fractional amount of hours as an integer multiple of ten-minute intervals.

Contrast that with:

In computing, floating-point arithmetic (FP) is arithmetic that represents real numbers approximately, using an integer with a fixed precision, called the significand, scaled by an integer exponent of a fixed base.

These two approaches are vastly different, so what are the pros and cons of each when choosing your language's default arithmetic?

The main upside of floating-point arithmetic is that it allows for reprensenting a wider range of magnitudes than a fixed-point number using the same number of bits, while also allowing for more relative precision for smaller numbers in the range. Another point in favour of floating-point arithmetic is that practically every computer supports it in hardware.

One downside of floating-point arithmetic for some applications, is that it's not cross-platform deterministic. That is, the same floating-point computation can have different results depending on

• Which processor is computing it;
• The rounding mode, which is a shared mutable state ─ if a third-party library sets a rounding mode then it will affect your code too;
• Whether the compiler decides to perform a + b * c as two instructions or a single fused multiply-add instruction;
• Whether or not the compiler decides to directly use intermediate results in a higher-precision 80-bit register, or truncate them to 64 bits by storing them somewhere else.

Some individual floating-point operations are deterministic ─ add, subtract, multiply, divide, and square root ─ because the specification requires these to give correctly-rounded results. But once you start doing multiple operations, or invoke transcendental functions like sin and cos, cross-platform determinism is no longer guaranteed.

This can be a problem for games or networked applications. We often want to minimise the amount of data sent over a network, or stored in a save or replay file; but this requires the program state to be recoverable by recomputing it from the data that is stored or transmitted. This could also theoretically be a problem in scientific computing, where different researchers may want to independently verify the result of a very long computation.

Another piece of baggage that comes with floating-point numbers is the "exotic" values of positive and negative infinity, negative zero, and NaN. Some high-level languages like Python protect the programmer from some of these, so that e.g. 1.0 / 0.0 raises an error instead of resulting in positive infinity, but this protection doesn't extend to overflows like 1e200 * 1e200, and detecting these could have a significant performance penalty.

The "contagious" nature of NaN arithmetic is particularly a problem for debugging: a long computation with many steps can produce NaN at the end, but this gives no indication of which step the problem occurred in.

Some problem domains, particularly audio processing, may favour fixed-point arithmetic because their outputs are supposed to be in a fixed-point format (i.e. audio samples with a given bit-depth). In these applications there is no need to represent audio samples with magnitudes outside of a known range, and there is not necessarily any benefit to having greater resolution for lower-magnitude samples.

All of that said, I think there is something in the question which is a bit of a false premise: the phrase "your language's default arithmetic" pre-supposes that either fixed-point or floating-point arithmetic must be privileged by the language in some way. But this is empirically not true, even in popular mainstream general-purposes languages ─ int is a fixed-point type, and integer arithmetic is fixed-point arithmetic.

So it's already totally normal for an operator like + to mean either fixed-point or floating-point arithmetic depending on the types of its operands. What's less typical is to have fixed-point fractional number types built into the language. But a language could easily have such types without having to choose whether those types or floating-point types are the "default"; the programmer already has to choose which numeric types to use in their programs.

Fixed point arithmetic is more consistent

Adding 0.128 to a number will always add 0.128 to the number until you reach the limit.

In floating point arithmetic the steps get more inconsistent the bigger the numbers get, leading to glitching. For example there exists a number such that x+1 == x due to rounding.

Floating point arithmetic can represent a wider range of values

Fixed point has a fixed accuracy, so either it's limited to storing only small values or it loses accuracy when the step size gets too big.

In a floating point number the range is dynamic. The same data type can store 0.000000232 and 1,123.512.4. This is very nice because you don't always know what range your data is going to be in.

When choosing the default for your language

Most languages use floats as default, so for consistency it has a huge benefit. The difference is rather small.

I'd choose Decimals as the default number type before I'd use fixed point numbers because if you are going to be inconsistent anyway use something that has a bigger benefit.

• Further to the first point: If the sum of many small numbers is a medium number, the sum of a huge number plus many small numbers minus a similar huge number can ignore the small numbers in floating point. The order in which numbers are added makes a difference to the result. BIG = 17.E25; SMALL = 100.; PRINT *, BIG+TINY-BIG produces 0, not 100. Commented May 19, 2023 at 19:12

Firstly, consider fixed-point decimal arithmetic vs. fixed of floating point binary: the former is suitable for precise representation of monetary values, while the latter shall never be used for such. So if you have to handle monetary values, this is your only reasonable option anyway. E.g., if your language is designed for financial applications, then of course your default data type must be decimal fixed point.

Now, the other case where fixed point binary is essential is high-level synthesis languages - e.g., targeting FPGAs or ASIC designs. Floating point arithmetic is very expensive, and in a lot of cases excessive, when the required precision is possible with simple fixed point instead.

In languages like COBOL or PL/I, there are a huge number of numeric types which all work similarly, but have different numbers of digits to the left and right of the decimal point. It would be common for code to add an "item price" value with e.g. (6,2) digits to an "order total" value with (8,2) digits; a compiler would need to be able to accommodate arithmetic with arbitrary combinations of data sizes.

In many cases, code which uses a variety of fixed-point types could be more efficient on machines with small word sizes than floating-point code would be, but only if a language implementation can conveniently handle operations that involve operations of different types. Nowadays, relatively few programs target the kinds of hardware where such types would be useful, and most compilers' type system logic is not well equipped to handle such types efficiently. If one were designing a language that was specialized for use on 8-bit targets, fixed-point support could be very useful, but I don't see much use for such types on larger platforms.

This isn't really a "pro-vs-con" situation.

Fixed-point and floating-point numbers have completely different areas of application.

Fixed point arithmetic is used for accountancy systems - money, stock, working hours, that kind of thing.

A characteristic of accountancy systems is that they are designed in the first place to handle numbers that are exact and discrete, and not arbitrarily divisible. In effect, accounting systems work only with integers, and all operations are algebraically reversible and fully conservative of magnitude.

It isn't strictly necessary that accounting systems are "fixed point" - that is, for their minimum quantum (or equivalently put, their scaling factor) to be defined at design time - but in practice they almost always are fixed point. This is because in the money realm coinage has a defined minimum unit, in the stock-keeping realm each type of good will have a defined minimum unit, and in the time-tracking realm, there will be a minimum unit like tenths of an hour.

Floating point arithmetic meanwhile is used to manipulate certain kinds of physical measurements, where it is usually more important that a wide range of quantities can be approximated rather than that all quantities and calculations are exact. A crucial property of floating-point numbers is that the preciseness of the approximation is related to the overall magnitude.

Typically a physical measurement process has a certain amount of error, and typically this error is not an absolute quantity but a proportion of the overall magnitude of the measurement, so floating point is intended to fit nicely with situations which have this property. Given a ton heap of sand, you don't know in the first place exactly how many atoms are present, and you certainly aren't concerned if an atom goes missing.

Operations on physical substances - such as a chemical process, or even just transportation - typically involve residuals and soiled containers and so on. So when recording or calculating these kinds of quantities, it isn't important to be totally error-free in the arithmetic, because the physical reality to which the records relate is not free of "error" (i.e. unmeasured byproducts, wastage, and so on).

This is very different from physical money, where coins don't lose any proportion of their value as they move between purses.

The choice between fixed-point and floating-point storage types then, is typically about identifying whether you are dealing with discrete quantities that require exact arithmetic, or only approximate quantities where error is proportional to the magnitude.

• There is also a lot of overlap between use cases though, for example a lot of multimedia stuff (audio and image manipulation) and many sensor readings are done in fixed-point. In those cases there was probably measurement error proportional to the magnitude, but fixed-point is used anyway, for different reasons.
– user1030
Commented Aug 4, 2023 at 20:11
• @harold, I'd certainly be interested to hear a more elaborate rationale for why fixed point is specifically chosen in those cases. Certainly my argument is not that you choose floating point simply whenever there can be measurement error - you choose floating point when exactness of arithmetic is not a required property (i.e. it's not an accounting system), but a larger overall range is desirable than for fixed point representations (with loss of precision proportional to the extremity of the value being an acceptable property). Commented Aug 4, 2023 at 21:37