17
$\begingroup$

This question is related to, but not the same as, What are some ways of reporting compilation errors? as that questions gets into the design of handling compiler errors at the compiler level (and the answer there gives personal examples of their approach to handling them). This question is more of a philosophical one regarding how far one needs to go when offering information about a compiler error.

Rust has a reputation for extremely helpful compiler error messages, going so far as to point out the specific portion of a line that's problematic as well as suggestions to fix the error. Those suggestions sometimes perfectly account for a programmer's intentions and offer correct solutions to them.

These advanced explanations are not very common in programming compilers, and when designing a programming language, I'm curious how important it would be to sink all that time into supporting such a robust compiler error handler, or whether the seemingly much more common approach of "Solid explanatory error message with supporting documentation" is sufficient.

To narrow this down a little bit further, my intent would be to target an audience that should, ideally, have a fair bit of programming knowledge from other languages prior to starting this language's use (i.e. I don't expect this to be that discoverable or as accessible to new programmers given its scope) and am weighing my options as it pertains to the work I would put into this sort of feature.

$\endgroup$
4
  • $\begingroup$ Python recently is getting Rust-esque error messages as well $\endgroup$
    – Seggan
    Commented May 18, 2023 at 14:14
  • 2
    $\begingroup$ "going so far as to point out the specific portion of a line that's problematic" is the minimum I would expect from any compiler that's meant to be used these days. It's not that hard to put a ^~~~~ below the errors in your compiler messages. Regarding suggestions, that could be done later and you could start with simple variable name typo suggestions. $\endgroup$
    – felipecrv
    Commented May 18, 2023 at 14:38
  • $\begingroup$ If you wonder whether they're worth the effort, just try to troubleshoot errors relating to, say, something going wrong with std::string, std::vector, and/or std::ostream in C++ (or with template types in general), or especially troubleshooting anything that involves SFINAE. Error messages can easily devolve into borderline gibberish if you don't know how templates work, and even then they can still be hard to parse, and that's with compilers trying to make them helpful. I'd hate to imagine what it'd be like with anything less helpful! $\endgroup$
    – Justin Time - Reinstate Monica
    Commented Jul 8, 2023 at 3:53
  • $\begingroup$ A counterexample: “A friend of mine in a compiler-writing class produced a compiler with one error message ‘You lied to me when you told me this was a program’.” — Pete Fenelon. And a more widely-available one: by default, the Unix line editor ed indicates all errors by displaying a single ?, as ‘the experienced user will know what is wrong’ (though later versions can show more descriptive error messages on request). $\endgroup$
    – gidds
    Commented Sep 22, 2023 at 12:11

5 Answers 5

Reset to default
16
$\begingroup$

For context, in case any readers are unfamiliar with Rust's error messages, they are very detailed. Here's a typical example; yes, this is the message for one error.

error[E0382]: borrow of moved value: `x`
 --> src/main.rs:5:15
  |
2 |     let x = String::from("Hello, world!");
  |         - move occurs because `x` has type `String`, which does not implement the `Copy` trait
3 |     drop(x);
  |          - value moved here
4 |     
5 |     println!("{x}");
  |               ^^^ value borrowed here after move
  |
  = note: this error originates in the macro `$crate::format_args_nl` which comes from the expansion of the macro `println` (in Nightly builds, run with -Z macro-backtrace for more info)
help: consider cloning the value if the performance cost is acceptable
  |
3 |     drop(x.clone());
  |           ++++++++

For more information about this error, try `rustc --explain E0382`.

You've got: an error code and short description, a source file reference, the source line where x is declared (plus explanation), the source line where x is moved (plus explanation), the source line where x is used after being moved (plus explanation), a note that this occurred in the expansion of a macro (plus a hint for how to debug within macros), a suggestion for how to fix the error (plus explanation), and a hint for how to get more information about the error.

One reason that the Rust developers might feel it's worth their time and effort to generate such detailed error messages, is that they want to make Rust a popular, general-purpose language, but the language itself has some features ─ particularly ownership, lifetimes, and traits ─ which don't exist in many other popular languages, so they are unfamiliar to most newcomers to the language. Even for some experienced programmers, learning Rust can feel like being a beginner again.

So Rust is something of a special case in this regard, because most languages either aren't radically different to what people are used to, only require the programmer to understand concepts that are well-explained in many sources, or are specialised languages aimed at programmers who mostly understand the necessary concepts already.

Therefore I would say it's worth going into that level of detail in your error messages if your language is aimed at people who don't already understand the core ideas that the language is based on, particularly if those core ideas are unusual compared to other languages. Even then, it's probably best to focus your effort on error messages where the error does relate to one of those specific concepts. The flipside is that long error messages can be annoying if they explain things the user already understands.

$\endgroup$
11
  • $\begingroup$ Making it toggleable might be cool...or even per error type. Maybe if you've seen the same error three times the compiler would start showing a minimized version of it unless you tell it not too (like how Zork shortens locations to just their names after you visit them the first time) $\endgroup$
    – rydwolf
    Commented Jul 1, 2023 at 16:26
  • $\begingroup$ I vehemently disagree with the reasoning. Popularity, maybe, exotic features, no. Suggestions for typos, for example, were already popular in Clang at the time, and that's got nothing to do with "exotic features". And today, many of the errors of rustc in which special effort was made to make the error extra clear have nothing to do with exotic features. I think it's really a matter of culture. There was a desire to make the life of developers easy, and a knee-jerk reaction to C++ horrors perhaps, and thus special attention was devoted to smooth the way as much as possible. $\endgroup$
    – Matthieu M.
    Commented Sep 12, 2023 at 15:20
  • $\begingroup$ @MatthieuM. Of course it's about making the life of developers easy. The point is that the way to do that depends on who the developers are and how well they understand the language. My life would not be made easy by having such verbose error messages in e.g. Java; that would make my life more difficult because I would have to consciously ignore most of the message. So that we're on the same page, a use-after-move error in Rust results in a 16-line error message, for just that one error. $\endgroup$
    – kaya3
    Commented Sep 12, 2023 at 16:51
  • $\begingroup$ As for culture as a reason, I don't think that's very helpful given that the question is "is it worth it?", not "why do the Rust devs do it?" ─ and it leaves open the question of why the Rust devs have instituted such a culture. $\endgroup$
    – kaya3
    Commented Sep 12, 2023 at 17:09
  • $\begingroup$ @kaya3: I've seen the example, thank you. I think it's important to break down the 17 lines, though. First, there's 3 sections: the main error section (10 lines), the suggestion section (5 lines), and the footer section (2 lines). This is important, because power users will quickly learn to "filter out" the sections they don't care about -- though an option to hide them by default would still be nice. Then, within the main error section you've got: 1 line with the error, 1 line with the source location, and 8 lines with annotated code, showing off the 3 relevant lines of your code, and... $\endgroup$
    – Matthieu M.
    Commented Sep 12, 2023 at 17:10
15
$\begingroup$

Let me put it this way...

Error messages are the user interface of a compiler

Except for language extensions, the language that a typical compiler accepts is more or less independent of the compiler. Programmers rarely modify command-line options once they are set. Almost none of a compiler's users will spend time inspecting the generated code.

So the main "user interface" of a compiler is its errors and warnings.

I once used a compiler (not naming any names) which would occasionally produce this output:

Your program contains one or more syntax errors.

And that's it. Obviously this is an extreme example, but it goes to show that a bad error message is useless.

High quality error diagnostics can save a programmer hours. The programmer can check the intent behind a piece of code, without worrying about syntax or type correctness, and the compiler can do the work that it's relatively easy for a compiler to do.

$\endgroup$
1
  • $\begingroup$ I've used an assembler that would produce an error file containing a list of all errors, but limit console output to an indication that errors had occurred. This allowed the file to contain more verbose reporting than would be practical on the console, but not interfere with users' ability to see other messages produced by other programs. $\endgroup$
    – supercat
    Commented Sep 26, 2023 at 21:46
14
$\begingroup$

It depends on the audience for your language

If your language is intended to be used by people already familiar with programming, and uses generally the same programming syntax as other languages, then it's probably not worth it.

On the other hand, if your language is intended to be used by novice programmers, or includes its own confusing syntax other programmers are likely to be unfamiliar with, then more detailed error reporting is helpful.

Remember, any time you invest in error reporting will get payed back several-fold overall by users spending less time debugging their code.

$\endgroup$
0
10
$\begingroup$

I built a compiler once that had these sorts of error messages, and I'd give them mixed reviews. After the low-hanging fruit, making things just a little better required a lot more work. They were often very helpful for illuminating unusual corners of the language, but the times when they were misleading were sometimes net harmful and hard to correct.

These error messages were added in a student project with a substantial amount of work in it — hundreds of hours — but not nearly as much as Rust will have had. This language was intended for teaching use, so not the same domain as you, but many of its users had significant prior programming experience, and it had some unusual design features that could take some explanation.

These error messages would identify specific parts across multiple lines that were relevant to the problem, describe in natural language what the expectation was, and why, and often provide a suggestion or two of what could have been intended.

In practice, they had mixed effectiveness: people were very favourable towards them and towards the language while using them, in both controlled user studies and ordinary use. We observed many places where the same error was not solved when using the more conventional compiler error messages that were implemented previously. The benefits were very clear.

The other side of that, though, was that people would also switch off when they weren't confident of what they were doing, and blindly follow any suggestion given even when it was directly counter to what they were trying to do. This was particularly the case when they were using those less-common language features, which is exactly the situation where the messages were also the most helpful.

These unproductive changes could lead them further and further away from the goal. It's an artifact of giving wrong suggestions or inapplicable explanations, which could be improved with more engineering effort — though at times there are just two equally-plausible candidates and people choose the wrong one. One major usage site disabled the improved messages entirely for this reason and settled for less constructive, but visibly unhelpful when unhelpful, feedback.

These sorts of errors are useful, but the marginal per-message improvement is very costly. Whether it's worth the effort depends on what else that effort could go into: producing one "very explanatory" message is probably not as helpful as making five other messages "adequate", but once everything is at a passable level then concrete diagnostic improvements are one of the most noticeable ergonomic improvements for a compiler. Making sure they're not-wrong is also more worthwhile than speculative improvements.

$\endgroup$
1
  • 4
    $\begingroup$ "but the marginal per-message improvement is very costly": it's notable that Rust has one very specific contributor, who has been relentlessly polishing error messages for the last decade. He regularly submits multiple small patches a week. So yes, the cost is staggering. I am very thankful to him, though :) $\endgroup$
    – Matthieu M.
    Commented Sep 12, 2023 at 15:23
7
$\begingroup$

There's a very obvious answer to this.

Yes, someone might end up spending what seems like far too much time writing the code that produces a single error message.
That could be many many hours, and really doesn't seem worth the effort.

But consider that this error message might be seen millions of times, and each time it is going to save time for the people doing the debugging.
Even if it saves each of them only a few seconds, it's already paid for itself many times over, and in reality it's going to save them a lot more time than that.

Just compare the time you'd spend in each of these two error messages:

  • Syntax error at line 86.
  • Syntax error detected at line 86: no matching closing brace for opening brace in column 23 on line 17.

It would take significant debugging time to determine what the "syntax error" is in the first case (especially when line 86 is where the parsing failed and the actual typo is many lines before that), while the second instantly reduces it to determining why the braces are mismatched.

$\endgroup$
5
  • 1
    $\begingroup$ I just ran into this: MSVC "not all control paths return a value" vs clang "enumeration value 'GTI' not handled in switch". $\endgroup$
    – Bbrk24
    Commented May 20, 2023 at 1:43
  • 6
    $\begingroup$ Even worse is that "syntax error at line 86" is often very much not an error at line 86, rather, line 86 is the point where the parser got stuck after blowing past the actual error many lines earlier. $\endgroup$
    – Jörg W Mittag
    Commented Jul 1, 2023 at 12:50
  • $\begingroup$ @JörgWMittag, thanks. I've incorporated that suggestion. $\endgroup$
    – Ray Butterworth
    Commented Jul 1, 2023 at 13:05
  • $\begingroup$ If it's possible that the typo was several lines earlier (e.g. because the language permits multi-line parenthesized expressions), the error message should hint at where to start looking. "Line 86, column 23: parenthesis opened at line 54, column 70 should have closed by now." $\endgroup$
    – Karl Knechtel
    Commented Jul 1, 2023 at 14:47
  • $\begingroup$ Another distinction is between compilers which would flag a precise location within a line, versus those that would merely flag a line. Trying to flag a location within a line would work better in languages with more concrete grammars than with others where constructs could mean different things based upon other constructs that could appear later, but something like Turbo Pascal 2.0 and 3.0 diagnostics worked pretty well given the simplicity of their implementation. $\endgroup$
    – supercat
    Commented Sep 12, 2023 at 15:03

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .