21
$\begingroup$

Intel created a series of floating-point coprocessors for the x86 called the x87. In addition to supporting IEEE single and double precision numbers, it also supported an 80-bit extended precision number. Some C compilers (e.g. clang) mapped this to the long double type in C, but others (e.g. MSVC) didn't.

Many languages have no built-in support for this type. The most recent example I know of that does is Swift, which has a Float80 type only available when compiling for Intel processors. (Swift also has CLongDouble which represents the exact type that the C compiler takes long double to mean, which is sometimes the same thing as Double.) The only time I've seen Float80 or long double used in practice is to use the increased precision to emulate a fused multiply-add instruction on older processors that don't support it, or very rarely to avoid loss of precision when converting from a 64-bit integer.

Evidently support for this type isn't strictly necessary, since so many languages get by without it, but why would a modern language choose to support or not support the type?

$\endgroup$
7
  • 3
    $\begingroup$ Related question from Stack Overflow $\endgroup$
    – dan04
    Commented Jul 15, 2023 at 15:54
  • 3
    $\begingroup$ As an aside, on arm64, long double is 128-bit (and implemented in software). $\endgroup$
    – jpa
    Commented Jul 16, 2023 at 8:50
  • 3
    $\begingroup$ Yes, even on Windows, GCC and clang apparently default to 80-bit long double, unlike MSVC. They have options -mlong-double-64/80/128 to change the ABI, making long double a different type, allowing compat with MSVC for example. Also, see Did any compiler fully use Intel x87 80-bit floating point? re: the (in)efficiency of 80-bit FP load/store on modern x86 (quite a bit slower than load/store with conversion from/to double or float) $\endgroup$ Commented Jul 16, 2023 at 14:00
  • 1
    $\begingroup$ C doesn't have 80 bit floating point types. It has float, double and long double, anyone of this may or may not use 80 bit. If you want a language similar to C, you need to support types with minimum requirements that may have different actually features on different platforms. $\endgroup$ Commented Jul 17, 2023 at 11:10
  • 1
    $\begingroup$ @12431234123412341234123 I never claimed that any type was specifically 80 bits. In fact, I said quite the opposite, that some compilers never supported 80-bit floating-point types in the first place. $\endgroup$
    – Bbrk24
    Commented Jul 17, 2023 at 18:14

5 Answers 5

20
$\begingroup$

I can see several reasons to not bother supporting it.

  • It's not supported by many processors other than x86. So if your language is going to be cross-platform, what do you do when building code for another platform? You could map it to float64, but if the programmer is using it, presumably it's because they really wanted the extra precision. You could provide software emulation, but that's a lot of trouble and will have unsatisfactory performance. You could map it to float128, but there are few if any FPUs with native float128 support so you are back to software emulation again.

  • It is only supported by x87 floating point, which is itself painful for a compiler writer. The stack-like register layout is awkward for register allocation and code generation. Until recently, gcc had a 3000-line source file in its "machine independent" layer, dedicated to handling register stacks, which AFAIK only ever applied to the x87.

    And since it's a "legacy" instruction set, modern CPUs don't tend to optimize x87 instructions very well. If you don't need float80, then you have the option to do all your x86 floating point with SSE, which is a much more "normal" architecture with a random-access register file (xmm), and ignore the x87 altogether. SSE is supported by all x86-64 CPUs, and by all 32-bit x86 CPUs from the last 20 years or so.

  • Even within x87, using float80 is more awkward than other types. It seems clear that the x87 design assumed that programmers would primarily use float32 and float64; float80 was intended more for "internal use", as an implementation detail to provide additional precision for intermediate results. So while you can use float80 explicitly, it comes with some restrictions. For instance, most two-operand x87 arithmetic instructions use the top of the floating-point stack st(0) as the implicit first operand, and the second can be either another st register, or else a float32 or float64 from memory - but not a float80. For float80, you need an additional instruction to load it into a register first. So you'll need more special cases in your code generation.

See also:

$\endgroup$
5
  • 4
    $\begingroup$ It´s not unique to x86. The Motorola 68881/68882 coprocessors support an 80-bit floating type as well. $\endgroup$
    – d3jones
    Commented Jul 16, 2023 at 0:56
  • 3
    $\begingroup$ x87 FP math instructions are not bad even on modern microarchitectures, and fxchg is still zero latency (although 2 uops since Haswell, up from 1). It's 80-bit load/store being slow that are the real killers for long double performance. Like fld m80 being 10 uops on Zen 4 with 1 per 16 cycle throughput (up from 4c on Zen 3, vs. Ice Lake being 4 uops / 2c). vs. 2/clock throughput for fld m32/m64. And fst m80 being 17 uops with 5c throughput on Zen 4, similar to Ice Lake's 5c throughput for several generations. $\endgroup$ Commented Jul 16, 2023 at 14:09
  • $\begingroup$ @PeterCordes: Hmm, where are you finding this data? uops.info doesn't seem to have info on x87 instructions for anything later than Skylake. But on Skylake, for instance, addsd executes on ports 0 or 1, so it gets double the throughput of fadd which only executes on port 5. $\endgroup$ Commented Jul 16, 2023 at 17:49
  • $\begingroup$ Agner Fog's tables have up to Ice Lake. Yes, fadd and fmul throughput are only 1/clock vs. 2/clock for SSE/AVX, but they are fully pipelined and 3 or 4 cycle latency respectively. In code with an even mix of add and multiply, you have a best-case throughput of 2 FLOP/clock with SSE or x87. In code with only one or the other, you have a factor of 2 advantage for SSE/AVX scalar math. That's not great, but my point was that it's only truly bad when we look at fld m80 / fstp m80 performance being (much) worse than the 1/clock or 2/clock we get with SSE/AVX or x87 m32 / m64. $\endgroup$ Commented Jul 16, 2023 at 17:56
  • $\begingroup$ Other fun facts from flipping through the table: (1) Sandybridge made fcomi 3 uops, up from 1, as legacy x87 became less relevant. (2) Skylake moved the x87 fadd/fcom unit to port 5 (previously port 1). So Skylake and later can do 3/clock scalar FP adds: two with SSE on p01, and one with x87 on p5. (3) fmul latency was 5 cycles until Ice lake; CPUs from Skylake to Cannon Lake has 5c fmul vs. 4c mulsd. $\endgroup$ Commented Jul 16, 2023 at 18:02
11
$\begingroup$

The 8087 had 80-bit registers so that if the inputs to your computation had 64-bit accuracy, the outputs would also have 64-bit accuracy.

The way floating-point arithmetic was supposed to work, when IEEE 754 and the 8087 were designed, is that when you compute something like w ← a + bx + cyz, all of the intermediate values are computed at a higher precision than the inputs and outputs. This is similar to the best practice for hand calculation. People sometimes ask "if I'm calculating a result to 3 sig figs, should I round all of the intermediates to 3 sig figs also?" and the answer to that is no—not if you can avoid it. Keeping extra digits around helps to avoid cumulative accuracy loss from roundoff.

Extra digits make it easier for ordinary mortals to write floating-point calculations that won't go wrong for hard-to-analyze reasons. The more extra precision you have, the more you can imagine that your custom formulas (a + bx + cyz) will behave similarly to library functions (sin x) that were designed by experts to have a error of no more than 1 ulp over their whole domain. You don't get guaranteed accuracy, but you do get more reliable accuracy.

As someone who doesn't know any of the black magic needed to implement those library functions, but does know enough about floating point (and sig figs) to understand the value of the extra digits, I would really appreciate having more programming languages that let me use them. It's especially maddening when languages like CPython, which are so inefficient in both space and time that the extra precision would cost essentially nothing, still don't let me use it.

$\endgroup$
4
$\begingroup$

Evidently support for this type isn't strictly necessary, since so many languages get by without it, but why would a modern language choose to support or not support the type?

A modern language may choose to support the notion of it. In other words, the language may have built-in support for a Float0 type, representing the more precise native hardware floating point available.1

The 0 suffix means that it has not a predefined size, but it is a floating point anyways. It can be converted between other number types in the number tower, and its bits could be converted from or into a byte[].

The marketing is easy: Float0, the best wider floating point representation that the language can find in your computer.

1. Or the software emulation configured in compiler parameters.

$\endgroup$
3
$\begingroup$

On many embedded platforms without floating-point units, computations using a 32-bit or 64-bit mantissa without an "implied 1" would be faster, more precise, and in just about every way better than those IEEE-754 64-bit double-precision values. Unfortunately, the way the C Standard added long double broke a key aspect of the language: that all floating-point values passed to variadic functions be converted to a common type.

If you're designing a language for use with embedded systems that may lack floating-point unit, I'd suggest having the following floating-point types:

  • float: typically 32 bits total, unless configured to be a different size.
  • long float: type to which float values are promoted for computations (and might be as small as float or as big as long double.
  • double: typically 64 bits total, unless configured to be a different size.
  • long double: type to which double values are promoted for computations.

I would then suggest having a means of explicitly passing types other than double to functions, but say that expressions that don't explicitly force the type of a floating-point value passed to a variadic function would by default be converted to double.

On many platforms, operations on 64-bit floating-point values would require unpacking the mantissa to a 64-bit combination of registers, performing computations with a 64-bit mantissa, and then packing the result. Processing an expression like a+b+c by unpacking a and b, adding them, normalizing and packing the result, and then unpacking that result along with c, adding them, and finally normalizing and packing that result, would take more work and yield less precise results than omitting the operations that would normalize and pack and then unpack the result from the original computation.

Extended floating-point types are sufficiently uncommon that most programmers wouldn't particularly favor them for their improved semantics, but if a set of development tools for an embedded system can perform floating-point calculations which are both faster and more precise than its competitors, that would be a nice advantage.

$\endgroup$
4
  • 5
    $\begingroup$ I would hope that a new language, if it is going to have variadic functions, would support them in a better way than the afterthought hack that is <stdarg.h> (and its predecessor <varargs.h>). So hopefully the type conversion issue wouldn't arise at all. $\endgroup$ Commented Jul 16, 2023 at 17:32
  • 1
    $\begingroup$ @NateEldredge: Even if a language has better calling convention for its own variadic functions (I could offer some ideas, if you'd want to hear them), it would still be useful to support interoperability with other languages that use the C calling convention for such functions. $\endgroup$
    – supercat
    Commented Jul 16, 2023 at 17:42
  • $\begingroup$ @supercat Swift supports calling C variadic functions via withVaList. I’ll note that it only supports calling e.g. vprintf and not printf in this manner. $\endgroup$
    – Bbrk24
    Commented Jul 16, 2023 at 17:44
  • $\begingroup$ @NateEldredge: I wish that compound literals had been specified as yielding static const lvalues when all values were compile-time constants, and non-l-values otherwise, but there was a convenient syntax to take a non-l-value, copy it to a temporary, and yield the address thereof, in sitautions where the resulting pointer is used in a dereferencing or function call expression. $\endgroup$
    – supercat
    Commented Jul 16, 2023 at 17:56
2
$\begingroup$

Are x87 long doubles still relevant?

No. The x64 instruction set deprecated the x87 FPU in favour of SSE for doing floating point math:

The x87, MMX, and 3DNow! instruction sets are deprecated in 64-bit modes. The instructions sets are still present for backward compatibility for 32-bit mode; however, to avoid compatibility issues in the future, their use in current and future projects is discouraged.

This means that, for x64 applications, there is no guarantee the x87 instruction set will be present.

In addition to supporting IEEE single and double precision numbers, it also supported an 80-bit extended precision number. Some C compilers (e.g. clang) mapped this to the long double type in C

It may be worth noticing that the C language standard is intentionally vague in defining the type for exactly this issue you're seeing.

double must have greater or equal precision as float. At no point it says one must be 64-bit and the other 32-bit precision.

It is also worth remembering that the x87 FPU had no ability to store the 80 bits into memory. Those extra 16 bits only lived in registers and were lost once they spill into memory. Its usefulness has always been limited.

why would a modern language choose to support or not support the type?

You will have to ask the compiler implementor why they included it.

Writing for Legacy is a thing. There are industries still using Windows XP for their QA software/hardware and some banks still run SW written in COBOL for Mainframes in the 70's.

Why they'd chose to use old SW/HW instead of modern ones needs to be evaluated on a case by case basis, but often the reason boils down to one of these three:

  1. "It's too expensive to migrate everything, so we paid someone to add functionality we needed to our current compiler"
  2. "We got that HW in a bargain"
  3. "Don't replace what ain't broken"
$\endgroup$
6
  • 2
    $\begingroup$ x86-64 didn't officially deprecate x87; Microsoft did. Maybe that's what you meant since you called it "x64", Microsoft's name for it. Intel and AMD haven't deprecated it. x86-64 does make SSE2 baseline, guaranteeing that a more efficient way to do scalar math is available, so x87 is obsolete but not deprecated in the sense of pending removal. Even Intel's proposed x86S (which removes 16-bit mode) doesn't propose removing it from the hardware $\endgroup$ Commented Jul 17, 2023 at 1:22
  • 1
    $\begingroup$ Rumors of Windows x64 not supporting MMX or x87 floating point are greatly exaggerated. MSVC removed MMX intrinsics, but the kernel's context-switch code still saves/restores x87 state so machine code made with other toolchains continues to work fine. $\endgroup$ Commented Jul 17, 2023 at 1:30
  • 1
    $\begingroup$ Despite what learn.microsoft.com/en-us/windows/win32/dxtecharts/… about future systems possibly not supporting MMX or x87, I find it implausible that future Windows would set control-register bits so x87 and MMX instructions faulted instead of working in 64-bit user-space. Intel and AMD are very unlikely to remove them in the forseeable future, although they may continue to get a bit slower as dedicated HW support for them is removed to simplify other parts of the CPU. $\endgroup$ Commented Jul 17, 2023 at 1:32
  • 1
    $\begingroup$ Some major software like FFmpeg's software decoder for h.264 (AVC) video uses hand-written MMX instructions, including in 64-bit mode, so it would break backwards compat with some commercially-relevant software; something vendors go to great lengths to avoid. Perhaps relevant, the SSE3 feature bit (CPUID) indicates support for x87 fisttp as well as a bunch of XMM instructions, although a hypothetical x86 without x87 could just not support that either. $\endgroup$ Commented Jul 17, 2023 at 1:37
  • 3
    $\begingroup$ the x87 FPU had no ability to store the 80 bits into memory. - Not true: fstp m80 and fld m80 are valid (but significantly slower than m32/m64). felixcloutier.com/x86/fst:fstp . fst m80 isn't valid, so you'd need to duplicate it first with fld st0 if you want a copy left in the x87 stack after storing. And you can't use an 80-bit long double as a memory source operand for any math instruction, e.g. fadd m32/m64 are valid but not fadd m80. See Nate's answer here and mine on Did any compiler fully use x87 80-bit FP? $\endgroup$ Commented Jul 17, 2023 at 1:40

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .