Are there any advantages of evaluating expressions differently in compile time and runtime?

Question

From https://rtfeldman.com/0.1-plus-0.2, on different floating point number semantics:

Go takes a different approach. When you write 0.1 + 0.2 in Go, that expression gets evaluated to 0.3 at compile time with no precision loss. If you evaluate 0.1 + 0.2 == 0.3, you'll get true. However, if you use variables instead of constants—that is, set a := 0.1 and b := 0.2 and print a + b—you'll once again see 0.30000000000000004. This is because Go evaluates variables at runtime differently from constants at compile time; constants use precise (but slower) base-10 addition, whereas variables use the same base-2 floating-point math as Python, Java, JavaScript, and so on.

To me, the same expression evaluating to different values in compile time and in runtime sounds like a bad idea, as it can make the same code behave differently and generate inconsistent or incompatible results depending on when it was evaluated.

Are there any advantages of doing this? Why does Go do it this way?

The only advantage that I can think of is: if I have a constant evaluator that works the same regardless of the target architecture, I can ship just one pre-compiled standard library (to some intermediate representation) for all targets, which can make cross compilation easier.

Note: the quote is about floating point semantics but I'm wondering about the general idea of having different semantics for expressions in compile time and runtime.

Answer 1

Evaluating the same expression differently at compile time versus runtime would indeed be horrendous behavior. But, that's not really what's going on here.

Go supports "untyped" floating point expressions. These are floating point expressions that are not associated with a specific, fixed precision representation like float32 or float64, and they are evaluated as if the underlying type had infinite precision.

(Or at least that's the idea. Go actually leaves some of the details as an "implementation restriction". Looking at the actual implementation for the "usual" compiler in src/go/constant/value.go, it looks like Go tries to use rational numbers where reasonable or "big floats" from math/big otherwise. So, 0.1, 0.2, and 0.3 are likely represented exactly as rational numbers, while sqrt(2) probably gets a big float representation).

Anyway, it turns out that only constant expressions are allowed to be "untyped" like this, and so such expressions can be and are evaluated at compile time, but this is an implementation detail. The bottom line is that the expression 0.3 - 0.1 - 0.2 in Go is an untyped expression that evaluates to an untyped floating point value that is precisely zero, while the expression a - 0.1 - 0.2 (for const a float64 = 0.3) is a float64 expression that evaluates to around -2.78e-17. They are different expressions (or, if you prefer, the "same" expression evaluated using two different floating point types), and whether they are evaluated at compile time or runtime, their values will be different.

This is not entirely without precedent. In Haskell, the numeric literal 0.3 is internally represented as a polymorphic function call that retains the full precision of the value independent of any specific representation:

fromRational (3 % 10) :: Num a => a

Consequently, if the Haskell expression 0.3 - 0.1 - 0.2, is evaluated in a context where the floating point type is Double, the exact representations of the literals will be converted to inexact Double representations, and the expression will evaluate to some non-zero fuzz (at compile time or runtime, depending on how smart the compiler is). On the other hand, if the expression is evaluated in a context where the floating point type is Rational (the type of rational numbers), the expression will evaluate to exactly zero:

{-# LANGUAGE NoMonomorphismRestriction #-}

main = do
  let expr = 0.3 - 0.1 - 0.2
  print (expr :: Double)             -- -2.78e-17
  print (expr :: Rational)           -- 0 % 1 (i.e., fraction 0/1)

If you're unfamiliar with Haskell, note that the :: syntax is not a "cast". It's a type signature providing guidance to the type checker to resolve the types -- not only of the overall expression expr -- but of all of the numeric literals and operators within that expression.

So, here, expr represents a "loosely typed" expression containing conceptually infinite precision numeric literals. When it is evaluated at a concrete type, the resulting computation may or may not be precise, depending on the nature of the concrete type.

The advantage in Haskell is that a literal like 0.3, which isn't intended to be tied to any specific numeric representation, can always be used to select the "best representation of 0.3" for whatever concrete type it is ultimately assigned, without the loss of precision that might take place if it was first converted to a double.

Answer 2

Numbers are the quintessential value objects. $1+1=2$ anywhere in the world, but numbers in computers work differently.

$1+1=2$ in u8 and u64, but $1+1=0$ in u1, for the same reason that $255+1=0$ in u8. That is, number realizations in computers do not follow the same rules and identities of pure abstract numbers of pure abstract mathematics.

In source code, constants are special. Because constant folding can generate huge savings by successive code reductions, to the point of rendering entire functions, even entire programs, into a constant result that executes no code at all, it's very profitable to pour a lot of effort into these optimizations.

This is where abstract numbers and constants meet. As constants can be folded in successive rounds, this also can be done for numbers and number operations. But it is necessary to always keep in mind the above notion, that operations on number realizations generate different results than may be expected from abstract numbers.

An if ( 0.1 + 0.2 == 0.3 ) will probably fail in any computer that follows IEEE 734, because some of these numbers are not representable in binary floating point format. A decimal number that is not representable is aproximated into another number, and because of that, decimal identities do not hold anymore.

As you can imagine, this is bad for constant folding and branch eliminations, as exemplified on the if() above, and will run against human intuition.

So there is an appeal to treat numeric constants differently than turning them into realized numbers immediately for operations. By treating these source numbers as ComptimeBigDecimal, or even better, as ComptimeFraction, it is possible to have intuitive, previsible and stable results for additions, subtractions and multiplications, and is also possible to minimize the imprecision accumulation of operations and to postnote imprecision of divisions to one last Fraction into number realization format.

But as you noted, it is bad that number expressions generate different results in compile time and run time.

If a compiler treats number constants differently in compilation time, it also should expose this functionality as a library, and by doing so, be possible to replicate the results of compile time in run time.

But none of this will change the reality of existence, and even preference, of realized number types, with all the imperfections noted above. Because these realizated number formats are fast, sometimes hardware fast, and no simulated integer/decimal/fractional code can reach this performance.

Beyond these concerns, it's advantageous to treat source code numbers differently from numbers realized at runtime, even in the absence of number operations.

This is an example I recently encountered in Python:

>>> from fractions import Fraction
>>> Fraction(1.65)
Fraction(3715469692580659, 2251799813685248)
>>> Fraction("1.65")
Fraction(33, 20)

Why did $1.65$ get converted to such a weird fraction, even though it can be easily converted to $100/65$? Because the decimal constant 1.65 in the source code was first converted to a double, and then this other number was precisely converted in a decimal fraction.

By simply delaying the source code number literal realization, it would be possible to code Fraction() to accept a comptime_decimal_literal, that in turn would result in both expressions above reaching an exact fraction of $1.65$, instead of the approximate fraction of $3715469692580659/2251799813685248$.

Answer 3

If it has advantages, a better approximation would be to have a new type for decimals up to any finite precision, or just fractions. The new type doesn't need to be restricted to compile time. Once it gets assigned to a float, it gets truncated.

I don't think the difference between compile time and runtime is a big problem, as people should already learn the floating-point accuracy problem, and if they don't, they would get a surprise sooner or later. It's less of a problem if we use a different type, as it has a clear semantic and doesn't really add anything special.

But there are some other possible surprises:

• Decimals have the implication of being possibly inaccurate. It could be automatically cast to floating points in some cases, where you would want to keep the original value for fraction number types. But you could solve it by adding a tag to the type to distinguish them, if your language also has fractions.
• Some functions might be much slower in arbitrary precision than floats. If you don't like distinguishing the functions by names, you could solve it by also matching the expected output type, or branching based on the value range. They may evolve into powerful features. Sometimes your language might already have them. But if it didn't, you may not think it is worth the extra work and complexity.
• It's not sure what to do with x + 0.1 + 0.2 == x + 0.3 where x is a float. You may define it to anything as long as it is consistent. But by the slight more difficulty to understand it, it is likely to ruin what you have thought that could make the language better in this way.

The advantage could be to make the computation more precise in compile time, where you would allow it to use more time, because the work is reused, and to keep the efficiency at runtime. But firstly, the little more precision usually just doesn't matter. Secondly, if you really want to do this, you could do more, by using faster computations in the most inner loops and more accurate computations elsewhere. It just isn't useful if you only do it in one unimportant case.

But I had another idea. I would advocate also supporting fixed-point numbers (fractions with constant denominator, and with easier representation) in newer practical languages, as it could reduce mistakes in many situations such as currency. If your language has fixed-point numbers, apparently it has to work like this. And if your language may add support for fixed-point numbers later, it would create a bigger surprise if it didn't.

In general, you should better distinguish them by the type, instead of compile time versus runtime. But it is understandable to do it in this way if the language don't have fixed-points yet, and you only leave the option for later.

Answer 4

One of the design goals of the Go language is to be easy of use. It aims to be a prototyping language like Python, but still remain efficient by having a compiler and static typing.

This feature fits perfectly in the spirit of a prototyping language: worry less about the small implementation details. By having 0.1 + 0.2 evaluate exactly to 0.3, the developer does not have to think about floating point arithmetic in this specific circumstance and this allows for faster prototyping.

In programming languages there is often the trade-off of between following the programmers intent and of being precise. On one end of the spectrum, Python tries to follow the programmers intent. When evaluating expressions, it tries to predict what the programmer wants, not what would be the most logical result. This greatly improves developing speed and developer experience (up to debate). The downside is that Python is slow and can be unpredictable.

On the other end is Rust. Rust is very precise: it evaluates according to strict rules. The end result is always predictable and any piece of code leaves little doubt about what exactly is happening under the hood. Because of this, Rust code is often fast, robust and reliable. But the downside is that it can take quite some effort to express the things you want using the - relatively - small space of valid rust.

So on the continuum between following intent and precision, this implementation detail is closer to following intent. It makes it harder to predict what the code does, but maybe we don't want to predict what the code does. We just want it to work. So is this is a good thing? I would say it depends use case in the same way that Python is great for specific purposes.