# Pros and cons of throwing an error versus returning a NaN value

So I've seen that many languages return a NaN value when calculating an indeterminate arithmetic expression (e.g. 0/0). Other languages, like Python, throw an error (but Python still has a NaN value returned by float('inf')-float('inf')). What are the pros and cons of each approach to indeterminate arithmetic expressions?

This question is distinct from What are different ways of handling runtime errors? as this question asks specifically about ways of handling indeterminate arithmetic expressions, as opposed to the language mechanism for runtime error handling in general. Plus, there is no mention of NaN in the other question.

• @mousetail it does not. There are no answers talking about returning a NaN value vs throwing an error for undefined expressions. May 24, 2023 at 10:01
• Returning NaN is just one example of a method to handle errors May 24, 2023 at 10:36
• @mousetail That other question is too broad to be useful, and focused on cases where the error/not-error is encoded separately from the real result. Exceptional floating point values are a distinctive case, among other reasons because the error/not-error information is almost always encoded together with the value, and there are semi-error cases like denormal numbers. May 25, 2023 at 6:42
• This is NOT a duplicate, stop closing my question! May 25, 2023 at 8:16
• FYI, float('nan') also returns the builtin 'NaN' in Python May 25, 2023 at 11:14

### The need to check for NaNs

NaNs are in fact rarely useful, and tend to indicate bad input or algorithm. Just passing this value along tends to lead to a knock-on error later where it'll be harder to see whence it originated. Or even worse, it'll not be detected at all, and will have unfortunate effects:

• Seems like it would be kinda funny when that happensâ€¦ May 27, 2023 at 11:30
• One situation where NaN could be useful in my opinion is if you have a big array of numbers, and a few missing or corrupted numbers in that array. When you have calculations to apply to all numbers in the array, you may not want the whole program to crash, and all calculations being interrupted or thrown out, just because of a few wrong numbers in the array. Imagine all numbers in the array are theoretically guaranteed to be nonnegative, but a few of them are slightly negative because of rounding errors. When you compute sqrt(x) for all x, the negatives are going to end up NaN.
– Stef
Jun 28, 2023 at 12:14
• @Stef I believe that would return a complex number in python. Jul 4, 2023 at 8:03
• @PlaceReporter99 Not really: both numpy.sqrt and pandas.Series.pow(1/2) would respect the type of the values: if they are already complex, it will use complexes, but if they are float, it'll put a nan for negative values. As for python's standard library functions, there are math.sqrt and cmath.sqrt, where the former will raise ValueError and the latter will use complex numbers, but those can only be applied to one number at a time, not to a whole array.
– Stef
Jul 4, 2023 at 9:40
• To be fair, this is more about UI libraries allowing the terrible practice of just slapping a float onto the screen. A UI application should format a number before rendering it (e.g. group thousands, localize decimal point/comma, handle thin non-breaking spaces for units, etc), and in languages where a float can be NaN, they should simply require providing a fallback string for those cases (i.e. treat it similar to how you'd have to format Haskell's Maybe or Rust's Some). Nov 22, 2023 at 9:27

# Performance

The IEEE 754 standard for floating point numbers defines, for example, that 0 / 0 should be a NaN. Many FPUs nowadays directly implement IEEE 754 operations, and therefore checking for the special condition of division by zero before calling the float-division instruction is less performant. Of course, such a difference is marginal in a language like Python, and it can freely choose whether throwing or NaNs are the most correct result.

As an example, this C++ code compiles to using the divsd instruction directly.

• Of course this depends on the situation. If you have a function like arr, x => arr.map(el => el/x) and you call it with x=0, it may be far faster to throw on the first division, depending on the size of the array. May 24, 2023 at 11:49
• You have a point. Maybe some compiler optimization can be found, but surely there are some places where throwing might be faster. May 24, 2023 at 15:50

One main reason for signalling arithmetic errors by NaN is the importance of array operations in scientific computing.

Let's say you want to take the logarithm of each element in an array. The result is another array, in which some values are possible NaN. This result is well-defined and in particular independent of implementation choices such as the order of iterations for multidimensional arrays. Moreover, you can filter out the NaNs, which is often a reasonable way to handle errors.

If instead the first negative argument to the logarithm raised an exception, the application programmer would have to implement complex error handling code in order to get all the "good" values in the array.

Put differently, NaN error values maintain the abstraction of arithmetic on arrays, rather than exposing the lower-level implementation detail of looping over elements.

IEEE-754 actually makes a number of affordances for different ways of handling erroneous conditions, most of which were sadly ignored or implemented poorly by programming languages; in short, it should be up to the programmer to decide on an application-specific basis, and adhering to what the standard actually says will get you there.

IEEE-754 specifies a number of exceptions. An exception can be signalled after performing an arithmetic operation; for example, the division-by-zero exception will be signalled if you (shocker) divide a (nonzero) number by zero, and the invalid operation exception will be signalled if you add infinities of opposite sign.

When an exception is signalled, the default behaviour is to set a corresponding status flag, compute an alternate result to use instead, and resume execution. For instance, following an invalid operation, you will get a nan, and following division by zero, you will get an infinity.

Already, we can see two different error-handling strategies. We could check if some result is nan. But we could also: clear all the status flags; perform some computation; check the value of the status flags. If you see that the invalid operation status flag got set, then you know that you performed an invalid operation somewhere.

But IEEE-754 also specifies some alternate exception behaviours that an implementation can optionally support. Two are of interest. You could have the operation throw an error (if your language supports error-throwing and unwinding); this is the other error-handling strategy you suggest. You could also specify some special code to be run when the exception is signalled, which would be able to examine the problematic operation and its arguments, and execute arbitrary code to compute an alternate result to be used instead (and continue operation). This could be useful if, for example, you think that 0/0 should be 0.

These two alternate exception-handling strategies could be unified into one if you support something like algebraic effect handlers or the common lisp condition system.

It is worth noting that, although x86 supports trapping exceptions, riscv does not, and on arm it is an optional feature implemented only by apple. This makes it difficult to implement them performantly and portably. One approach is as follows: execute a large chunk of straight-line math code with no side effects as-is, then check the status flags. If a status flag got set that is associated with alternate exception-handling behaviour, then re-execute that code with special instrumentation checking for exceptions after each operation.

• IEEE-754 also defines a concept of "signalling" vs "quiet" NaNs -- a "signalling" NaN is basically throwing an exception that aborts the current compilation. Support for signalling NaNs is optional, though. Aug 15, 2023 at 23:55
• one thing to point out is that the lack of language is largely the fault of ieee and CPU designers. the modes are specified by global CPU flags rather than per operation which makes composing different modes in programs very hard to do well Aug 17, 2023 at 0:11
• Explicit dataflow would be valuable, but I'm not asking the languages to support anything more than ieee-754 and the cpu architectures support. Aug 17, 2023 at 6:33

This doesn't apply to just division by zero.

First, a quick note on terminology. IEEE-754 distinguishes an exception from a trap. An exception is an event that may occur, and a trap is one possible response to that condition occurring.

The IEEE-754 standard defines five exceptions:

• Invalid operation: mathematically undefined (e.g. $$\sqrt{-1}$$).
• Division by zero: an operation on finite operands gives an exact infinite result. The 1985 revision only defined this for actual division by zero, but later revisions include things like $$\log 0$$.
• Overflow: a finite result is too large to be represented accurately.
• Underflow: a result is so small that it is outside the normal range.
• Inexact: the exact unrounded result is not representable exactly.

The first three are called "common exceptions", and usually can't be ignored when they occur. The last two, despite not being "common", are in fact extremely common: almost all floating point operations "raise" those exceptions and they can usually be ignored. It's actually quite rare that (say) a division operation returns an exact result.

The IEEE-754 standard specifies that whether or not each kind of exception actually causes a trap can be separately controlled. If the exception's trap is enabled, and the exception occurs, then a platform-specific stuff trap occurs (e.g. SIGFPE on Unix). If the exception's trap is disabled, then a "reasonable" value is returned and a flag is set recording that the exception occurs.

The exact value returned depends on a bunch of factors:

• Invalid operation returns a quiet NaN.
• Division by zero returns positive or negative Inf.
• Overflow returns either Inf, or the representable number of largest magnitude, depending on the rounding mode.
• Underflow returns a subnormal number or zero (which is technically just a kind of subnormal number), respecting the rounding mode.
• Inexact returns a rounded value.

When you raise a trap, there is then the question as to how this trap is reported. This is an especially important question for automatic vectorisation; suppose you have three parallel division operations that the compiler decides could be performed in a 4-way SIMD unit. Well that's nice, but if division by zero traps are turned on, you had better guarantee that the "unused" SIMD lane contains values that can't trap.

Some architectures, such as Cray-1 and Alpha AXP, made the design decision that, even when programmers want trapping arithmetic exceptions, it's a rare occurrence. Therefore, arithmetic exceptions don't necessarily need to trap in the precise location where they occurred in release code.

The Alpha has a "trap barrier" instruction TRAPB which checks the trap flags to see if a trap occurred and raise it if appropriate. For debug-mode compilation, the compiler might insert more TRAPB instructions than it would in a release-mode compilation. This decision makes for a much simpler superscalar CPU, and gives much better power performance, despite not being technically IEEE-754 compliant.

Note that this behaviour fits with the way that programming languages typically define trapping arithmetic handlers: a trap is raised by a high-level language concept such as a statement or a block, rather than a CPU instruction.

• $\sqrt{-1}$ is actually the imaginary unit $i$. Aug 16, 2023 at 7:14
• @TheEmptyStringPhotographer First off, in $\mathbb{C}$, there is no principal square root of $-1$. But more importantly, $i$ is not representable as a IEEE-754 floating point number. Introducing +Inf, -Inf, and NaN makes the floating point system algebraically closed without introducing complex numbers. Aug 16, 2023 at 11:46

## NaN as Null

As other answers already mentioned, NaN is about performance with floating point numbers. And the thing with fp math, is that when you use it, you use a lot of it, and then fp math performance is paramount.

You can think of NaN as the equivalent of float? type, that is, nullable number, with null replacing a failed computation. Making every math fp operation accept and return a float? may sound like an overkill, but that is what effectively happens with IEEE-754 and NaN, as an arbitrary computation may accept and return NaNs.

You can lessen the aNaNoyances of NaN by making this float? aspect a little more explicit. For example, in all math contexts, the signature of arguments and returns are always float?, a null float internally represented as a NaN, and where you need to show or store a represented valid computation, you will need to declare or cast this usage as no null, no NaN, it is really a number with digits, float.

## Configurable division

On ints, there is a lot of baggage, and on division in particular, a lot of rounding and failure modes. Instead of trying to convey them all in syntax, you may adopt a configurable approach.

Much of the performance cost associated with exceptions not comes from the need to test for the exceptional condition, but rather with the fact that actions which would otherwise be easily recognizable as having no side effects beyond yielding some kind of result, and thus having no side effects in cases where the result is ignored, must all be treated as having potential side effects.

Although a carefully designed approach using latching overflow flags with loosely defined semantics in some situations (e.g. including separate "has an overflow definitely happened" and "has an overflow definitely not happened" checks, where the former could be moved ahead of operations that might cause overflow but the latter could not) could probably offer better performance than NaNs, NaNs are probably the best approach among those that has been widely adopted for dealing with computations which are found to be erroneous.

On a language level, considering NaN as error and throwing an exception is a massive issue if you want to implement a component that simply passes values. There, a NaN can be a well-defined input and working around such issues is very very time consuming.