The main upside of floating-point arithmetic is that it allows for reprensenting a wider range of magnitudes than a fixed-point number using the same number of bits, while also allowing for more relative precision for smaller numbers in the range. Another point in favour of floating-point arithmetic is that practically every computer supports it in hardware.
One downside of floating-point arithmetic for some applications, is that it's not cross-platform deterministic. That is, the same floating-point computation can have different results depending on
- Which processor is computing it;
- The rounding mode, which is a shared mutable state ─ if a third-party library sets a rounding mode then it will affect your code too;
- Whether the compiler decides to perform
a + b * c as two instructions or a single fused multiply-add instruction;
- Whether or not the compiler decides to directly use intermediate results in a higher-precision 80-bit register, or truncate them to 64 bits by storing them somewhere else.
Some individual floating-point operations are deterministic ─ add, subtract, multiply, divide, and square root ─ because the specification requires these to give correctly-rounded results. But once you start doing multiple operations, or invoke transcendental functions like
cos, cross-platform determinism is no longer guaranteed.
This can be a problem for games or networked applications. We often want to minimise the amount of data sent over a network, or stored in a save or replay file; but this requires the program state to be recoverable by recomputing it from the data that is stored or transmitted. This could also theoretically be a problem in scientific computing, where different researchers may want to independently verify the result of a very long computation.
Another piece of baggage that comes with floating-point numbers is the "exotic" values of positive and negative infinity, negative zero, and NaN. Some high-level languages like Python protect the programmer from some of these, so that e.g.
1.0 / 0.0 raises an error instead of resulting in positive infinity, but this protection doesn't extend to overflows like
1e200 * 1e200, and detecting these could have a significant performance penalty.
The "contagious" nature of NaN arithmetic is particularly a problem for debugging: a long computation with many steps can produce NaN at the end, but this gives no indication of which step the problem occurred in.
Some problem domains, particularly audio processing, may favour fixed-point arithmetic because their outputs are supposed to be in a fixed-point format (i.e. audio samples with a given bit-depth). In these applications there is no need to represent audio samples with magnitudes outside of a known range, and there is not necessarily any benefit to having greater resolution for lower-magnitude samples.
All of that said, I think there is something in the question which is a bit of a false premise: the phrase "your language's default arithmetic" pre-supposes that either fixed-point or floating-point arithmetic must be privileged by the language in some way. But this is empirically not true, even in popular mainstream general-purposes languages ─
int is a fixed-point type, and integer arithmetic is fixed-point arithmetic.
So it's already totally normal for an operator like
+ to mean either fixed-point or floating-point arithmetic depending on the types of its operands. What's less typical is to have fixed-point fractional number types built into the language. But a language could easily have such types without having to choose whether those types or floating-point types are the "default"; the programmer already has to choose which numeric types to use in their programs.