Pros and cons of throwing an error versus returning a NaN value

Question

So I've seen that many languages return a NaN value when calculating an indeterminate arithmetic expression (e.g. 0/0). Other languages, like Python, throw an error (but Python still has a NaN value returned by float('inf')-float('inf')). What are the pros and cons of each approach to indeterminate arithmetic expressions?

This question is distinct from What are different ways of handling runtime errors? as this question asks specifically about ways of handling indeterminate arithmetic expressions, as opposed to the language mechanism for runtime error handling in general. Plus, there is no mention of NaN in the other question.

Answer 1

The need to check for NaNs

NaNs are in fact rarely useful, and tend to indicate bad input or algorithm. Just passing this value along tends to lead to a knock-on error later where it'll be harder to see whence it originated. Or even worse, it'll not be detected at all, and will have unfortunate effects:

Answer 2

Performance

The IEEE 754 standard for floating point numbers defines, for example, that 0 / 0 should be a NaN. Many FPUs nowadays directly implement IEEE 754 operations, and therefore checking for the special condition of division by zero before calling the float-division instruction is less performant. Of course, such a difference is marginal in a language like Python, and it can freely choose whether throwing or NaNs are the most correct result.

As an example, this C++ code compiles to using the divsd instruction directly.

Answer 3

One main reason for signalling arithmetic errors by NaN is the importance of array operations in scientific computing.

Let's say you want to take the logarithm of each element in an array. The result is another array, in which some values are possible NaN. This result is well-defined and in particular independent of implementation choices such as the order of iterations for multidimensional arrays. Moreover, you can filter out the NaNs, which is often a reasonable way to handle errors.

If instead the first negative argument to the logarithm raised an exception, the application programmer would have to implement complex error handling code in order to get all the "good" values in the array.

Put differently, NaN error values maintain the abstraction of arithmetic on arrays, rather than exposing the lower-level implementation detail of looping over elements.

Answer 4

IEEE-754 actually makes a number of affordances for different ways of handling erroneous conditions, most of which were sadly ignored or implemented poorly by programming languages; in short, it should be up to the programmer to decide on an application-specific basis, and adhering to what the standard actually says will get you there.

IEEE-754 specifies a number of exceptions. An exception can be signalled after performing an arithmetic operation; for example, the division-by-zero exception will be signalled if you (shocker) divide a (nonzero) number by zero, and the invalid operation exception will be signalled if you add infinities of opposite sign.

When an exception is signalled, the default behaviour is to set a corresponding status flag, compute an alternate result to use instead, and resume execution. For instance, following an invalid operation, you will get a nan, and following division by zero, you will get an infinity.

Already, we can see two different error-handling strategies. We could check if some result is nan. But we could also: clear all the status flags; perform some computation; check the value of the status flags. If you see that the invalid operation status flag got set, then you know that you performed an invalid operation somewhere.

But IEEE-754 also specifies some alternate exception behaviours that an implementation can optionally support. Two are of interest. You could have the operation throw an error (if your language supports error-throwing and unwinding); this is the other error-handling strategy you suggest. You could also specify some special code to be run when the exception is signalled, which would be able to examine the problematic operation and its arguments, and execute arbitrary code to compute an alternate result to be used instead (and continue operation). This could be useful if, for example, you think that 0/0 should be 0.

These two alternate exception-handling strategies could be unified into one if you support something like algebraic effect handlers or the common lisp condition system.

It is worth noting that, although x86 supports trapping exceptions, riscv does not, and on arm it is an optional feature implemented only by apple. This makes it difficult to implement them performantly and portably. One approach is as follows: execute a large chunk of straight-line math code with no side effects as-is, then check the status flags. If a status flag got set that is associated with alternate exception-handling behaviour, then re-execute that code with special instrumentation checking for exceptions after each operation.

Answer 5

This doesn't apply to just division by zero.

First, a quick note on terminology. IEEE-754 distinguishes an exception from a trap. An exception is an event that may occur, and a trap is one possible response to that condition occurring.

The IEEE-754 standard defines five exceptions:

• Invalid operation: mathematically undefined (e.g. $\sqrt{-1}$).
• Division by zero: an operation on finite operands gives an exact infinite result. The 1985 revision only defined this for actual division by zero, but later revisions include things like $\log 0$.
• Overflow: a finite result is too large to be represented accurately.
• Underflow: a result is so small that it is outside the normal range.
• Inexact: the exact unrounded result is not representable exactly.

The first three are called "common exceptions", and usually can't be ignored when they occur. The last two, despite not being "common", are in fact extremely common: almost all floating point operations "raise" those exceptions and they can usually be ignored. It's actually quite rare that (say) a division operation returns an exact result.

The IEEE-754 standard specifies that whether or not each kind of exception actually causes a trap can be separately controlled. If the exception's trap is enabled, and the exception occurs, then a platform-specific stuff trap occurs (e.g. SIGFPE on Unix). If the exception's trap is disabled, then a "reasonable" value is returned and a flag is set recording that the exception occurs.

The exact value returned depends on a bunch of factors:

• Invalid operation returns a quiet NaN.
• Division by zero returns positive or negative Inf.
• Overflow returns either Inf, or the representable number of largest magnitude, depending on the rounding mode.
• Underflow returns a subnormal number or zero (which is technically just a kind of subnormal number), respecting the rounding mode.
• Inexact returns a rounded value.

When you raise a trap, there is then the question as to how this trap is reported. This is an especially important question for automatic vectorisation; suppose you have three parallel division operations that the compiler decides could be performed in a 4-way SIMD unit. Well that's nice, but if division by zero traps are turned on, you had better guarantee that the "unused" SIMD lane contains values that can't trap.

Some architectures, such as Cray-1 and Alpha AXP, made the design decision that, even when programmers want trapping arithmetic exceptions, it's a rare occurrence. Therefore, arithmetic exceptions don't necessarily need to trap in the precise location where they occurred in release code.

The Alpha has a "trap barrier" instruction TRAPB which checks the trap flags to see if a trap occurred and raise it if appropriate. For debug-mode compilation, the compiler might insert more TRAPB instructions than it would in a release-mode compilation. This decision makes for a much simpler superscalar CPU, and gives much better power performance, despite not being technically IEEE-754 compliant.

Note that this behaviour fits with the way that programming languages typically define trapping arithmetic handlers: a trap is raised by a high-level language concept such as a statement or a block, rather than a CPU instruction.

Answer 6

NaN as Null

As other answers already mentioned, NaN is about performance with floating point numbers. And the thing with fp math, is that when you use it, you use a lot of it, and then fp math performance is paramount.

You can think of NaN as the equivalent of float? type, that is, nullable number, with null replacing a failed computation. Making every math fp operation accept and return a float? may sound like an overkill, but that is what effectively happens with IEEE-754 and NaN, as an arbitrary computation may accept and return NaNs.

You can lessen the aNaNoyances of NaN by making this float? aspect a little more explicit. For example, in all math contexts, the signature of arguments and returns are always float?, a null float internally represented as a NaN, and where you need to show or store a represented valid computation, you will need to declare or cast this usage as no null, no NaN, it is really a number with digits, float.

Configurable division

On ints, there is a lot of baggage, and on division in particular, a lot of rounding and failure modes. Instead of trying to convey them all in syntax, you may adopt a configurable approach.

Answer 7

Much of the performance cost associated with exceptions not comes from the need to test for the exceptional condition, but rather with the fact that actions which would otherwise be easily recognizable as having no side effects beyond yielding some kind of result, and thus having no side effects in cases where the result is ignored, must all be treated as having potential side effects.

Although a carefully designed approach using latching overflow flags with loosely defined semantics in some situations (e.g. including separate "has an overflow definitely happened" and "has an overflow definitely not happened" checks, where the former could be moved ahead of operations that might cause overflow but the latter could not) could probably offer better performance than NaNs, NaNs are probably the best approach among those that has been widely adopted for dealing with computations which are found to be erroneous.

Answer 8

On a language level, considering NaN as error and throwing an exception is a massive issue if you want to implement a component that simply passes values. There, a NaN can be a well-defined input and working around such issues is very very time consuming.