7
$\begingroup$

Many languages have large amounts of syntactic sugar. For example, while working with Python there are seemingly many ways to express the same operations.

For example, in list comprehension

[x for x in range(10)]

can be used rather than

li = []
for i in range(10):
    li.append(i)

I was going through the Python Full Grammar Specification and it just seemed so long and unwieldy. The Zen of Python itself mentions that "Readability counts." and "There should be one-- and preferably only one --obvious way to do it." This is seemingly violated with multiple syntaxes for 1 operation.

I know that languages tend to accumulate bloat and sugar over time, but what are the exact reasons for which this sugar should be considered and subsequently introduced.

$\endgroup$
2
  • $\begingroup$ If the sugar can do something unique, that cannot be done otherwise. For example x += 1 is the only way to add 1 to x without evaluating x more than once. (Besides x -= -1;) $\endgroup$
    – CPlus
    Commented May 17, 2023 at 5:26
  • 3
    $\begingroup$ I don't think list comprehensions are syntactic sugar of a for loop and append, it's more like syntactic sugar of map and lambda. $\endgroup$
    – naffetS
    Commented May 19, 2023 at 17:40

3 Answers 3

12
$\begingroup$

Syntax sugar means language constructs which are (or can be) implemented in terms of other language constructs, rather than directly in the host language. Syntax sugar serves at least three purposes:

  1. It reduces the complexity of the backend by implementing more language features in the frontend.
  2. It adds expressiveness to the language for relatively little implementation effort,
  3. It removes some of the need for excessive boilerplate code and inscrutable idioms.

On the implementation complexity: the backend of a compiler is typically the most cognitively complex part, since it does the necessary checks and analyses for program correctness, and also has to transform the program from one format (the AST) to another (the IR, bytecode, or other target language) while preserving the program's meaning. Each analysis or pass the backend does, you will typically have to walk the AST and do something for each node. So the fewer different kinds of node, the easier the backend will be to write.

Because syntax sugar can be implemented in the frontend (by desugaring), it allows language constructs to be added to the grammar without adding more kinds of nodes to the AST. For example, if you already have assignment expressions and a "comma operator", then you can implement let-in expressions by transforming let A = B in C into (A=B, C) in the frontend, and then there is no need for a distinct LetIn node in the AST. Or if you already have lambda expressions then you can implement let-in as (lambda A: C)(B).


On expressiveness, of course the list comprehension and the for loop in your example achieve the same result, but the list comprehension is clearly preferable for most programmers: it is one line instead of three, and it can be used in place of an expression where the for loop must be a statement. So you can write a list comprehension directly as a function's argument, or the body of a lambda.

Yes, this means there is more than one way to write the same thing, but there is one "obvious" way to write it; the other way is "obviously" inferior. In general, syntax sugar always means there is an alternative way of writing it, i.e. the syntax that it's "sugar" for, also known as the "desugared" form. But when the sugar is significantly neater or more convenient, it has clear benefits, and the cost is relatively low since it can be implemented by transforming the AST into the desugared form, rather than adding more code generation logic to the backend.

I should note that Python's list comprehensions aren't exactly syntax sugar; they aren't implemented by transforming the AST, and they don't behave exactly like a standard for loop in some regards. For one, they are executed in a separate stack frame.


On boilerplate and idioms, once a language becomes more widely used, these will start showing up. To define terms, "boilerplate" means a standard form of code that must be written very similarly each time to do a menial task, and an "idiom" is the use of a language construct for something other than its main purpose. It's not necessarily obvious when initially designing the language what boilerplate and idioms you're encouraging, but over time you'll start to see certain ways that the language can be used to solve certain problems, repeated over and over.

So a good time to add syntax sugar to a language is after you have some experience with using the language yourself, or seeing other people's use of the language. By observing what the boring or obtuse parts about using your language are, you can find out what tasks need more concise or clearer syntax. If your project is open source or you have a public issue tracker, requests for syntax sugar may also come in directly from users, and these may be some of the best effort-to-reward improvements you make to your language.

$\endgroup$
3
  • $\begingroup$ I'm not completely sold on having syntax sugar converted before the AST is built. It's going to mess up with code formatters/pretty printers, and error message may refer to AST node types that are not actually present ("error in the tuple at line 3? But there's no tuple there!"). $\endgroup$
    – BoppreH
    Commented Jun 30, 2023 at 12:41
  • 2
    $\begingroup$ @BoppreH Code formatters and pretty printers don't use the AST, because the AST has already lost information ─ particularly comments and intentional line breaks/spacing in the source code, potentially also the order of things which might be represented in a dictionary or bitfield. So these tools generally work with a different representation, like a concrete syntax tree (CST). Regarding error messages, this usually isn't a problem in practice because any synthetic AST nodes you create generally aren't the source of errors, but if you do get misleading error messages because of this then ... $\endgroup$
    – kaya3
    Commented Jun 30, 2023 at 13:25
  • $\begingroup$ ... there are ways to address it, but perhaps you should ask it as a separate question. $\endgroup$
    – kaya3
    Commented Jun 30, 2023 at 13:25
3
$\begingroup$

Syntactic sugar is a violation of orthogonality, so it should be used sparingly. It's most useful when:

  1. It can be applied to a variety of scenarios (e.g., not just "cryptographers writing bit-flipping functions", unless that's your language niche).
  2. It comes up often.
  3. Its representation hints at its meaning.
  4. The alternative way of writing it is:
    • surprisingly verbose,
    • requires making unimportant decisions,
    • or makes the source code ugly (adding indentation levels, line noise, etc).

To take the Python list comprehension example, in a more complete context:

send_email([u.email for u in users if u.is_active])

vs

active_users = []
for user in users:
    if user.is_active
        active_useres.append(user.email)
send_email(active_users)
  1. Applicable to multiple scenarios? Definitely.
  2. Comes up often? Yep.
  3. Clear meaning? I'd say so.
  4. Is the alternative undesirable? Yes on verbosity and unimportant decisions. The comprehension is an expression, and hence can be done inline; while the for-loop is four statements. And we had to pick a name for the temporary variable, which is famously hard (it should probably be active_users_emails instead).

And stretching the definition of syntactic sugar a bit, Rust's ? operator is also a good example.

On the side of bad examples, I'd vote for Bash's variable expansions:

$var
${var}
"$var"
"${var}"

Out of the four, only the last one does (mostly) what people want 99% of the time, and it's the ugliest. The quotes change the behavior of the expansion in non-obvious ways, and all of them have severe shortcomings, like expanding to empty strings when undefined.

$\endgroup$
1
  • $\begingroup$ FWIW, Rust's ? operator can be exactly implemented as syntax sugar: expr? is equivalent to match expr { Ok(x) => x, Err(e) => return Err(e.into()) }. So this isn't stretching the definition, I don't think. $\endgroup$
    – kaya3
    Commented Jan 25 at 21:40
2
$\begingroup$

I wouldn't call the given example "sugar". The comprehension tells the interpreter what result you want, while the for-loop tells the interpreter how to generate that result.

The results might be the same, but the comprehension gives the interpreter far more latitude in how it does it. In a more complicated example, the comprehension could be far more efficient than a loop.

There really is "only one obvious way to do it"; for anyone fluent in the language the loop would not be an obvious way of doing it.

It's similar to:

n1 = x * 4;
n2 = x + x + x + x;

where the first is what you want, and the second specifies how to calculate it. The addition method generates the same result, but it is not an obvious way of doing this calculation.

The interpreter might even calculate n1 as x<<2. which is faster than a multiplication.


In C, a good example of sugar is strings.

auto char s1[] = "sugar";
auto char s2[] = { 's', 'u', 'g', 'a', 'r', '\0' };

s1 and s2 are identical objects (arrays of characters of length 6) with identical values.

Using the quotation syntax makes it a lot easier to write and read the code. Even though it means exactly the same thing to the compiler, this unnecessary syntax is provided to make life easier for the humans. Except for in unusual circumstances, the quotation syntax is the one obvious way that everyone will use.

Consider something like:

r1 = f(a1, a2, a3);
r1 = call f with a1, a2, and a3;

Anyone fluent in the language would see no significant benefit to introducing the second syntax, and would never use it. It is totally unnecessary.

$\endgroup$
5
  • $\begingroup$ "the for-loop tells the interpreter how to generate that result." ─ this is sort of true, but a language implementation is generally free to transform programs however it likes, so long as the observable behaviour is the same. For example, the loop x = 0; for i in range(10000): x += i; could very well be replaced with the statement x = 49995000. An interpreter which does that would not be breaking any rules, but it also wouldn't be executing the for loop the way the code told it to. $\endgroup$
    – kaya3
    Commented May 20, 2023 at 5:15
  • $\begingroup$ The C example isn't quite sugar. While they may both work for char s[], only the string literal works for const char *s. $\endgroup$
    – Bbrk24
    Commented Jun 30, 2023 at 14:52
  • $\begingroup$ @Bbrk24 In some cases you can use a compound literal with char *s, but there are lifetime considerations. $\endgroup$
    – Barmar
    Commented Jun 30, 2023 at 23:53
  • $\begingroup$ @kaya3-supportthestrike Python is a tricky example, since x += i is sugar for x.__iadd__(i) $\endgroup$
    – Warbo
    Commented Aug 4, 2023 at 10:03
  • $\begingroup$ @Barmar: Compound literals in C99 are syntactic salt (something that looks like sugar, but is generally inferior to the replaced construct) for named objects whose lifetime are unambiguously determined by location and storage class. $\endgroup$
    – supercat
    Commented May 31 at 22:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .