26
$\begingroup$

Are there good reasons to attempt to keep the number of keywords/reserved tokens in a language to a minimum? Such as by repurposing existing keywords for new syntax instead of adding new ones. Related: Why do some programming languages choose to have a dedicated keyword for elseif instead of using else + if like in C?

I suspect that more keywords may make implementing a compiler more complicated, slower as there are more keywords to compare tokens against or similar. But are these concerns legitimate and are there other advantages to having/maintaining a low number of keywords?

$\endgroup$
1
  • 6
    $\begingroup$ Try to remember (or in the first place, learn) all the keywords in SQL! $\endgroup$
    – Bergi
    Jul 5, 2023 at 4:45

9 Answers 9

31
$\begingroup$

Historical Baggage

If you've got a language that's already been released, then adding a new keyword is almost always a breaking change, since anyone who happened to use that name as a variable or function is now going to get a parse error.

Bjarne Stroustrup has stated that the bizarre pure virtual function syntax in C++ was designed in that way to avoid having to convince the committee to add a new keyword "pure" to the language. Instead, we got

virtual void example() = 0;

Some languages get around this by defining "soft keywords". For instance, the new match / case statement in Python only treats the keyword specially when appearing as part of the statement, so match = 1 is still valid and creates a variable called match. But this massively complicates parsing.

$\endgroup$
7
  • 6
    $\begingroup$ I would guess that almost all of the reason for aversion to adding new keywords comes down to this. In a new language with no backwards compatibility concerns, nobody really cares that much about minimising the number of keywords. Any feature that might require a keyword is no easier to memorise by reusing a keyword from another feature (if anything, probably the opposite). Ditto performance/complexity of the language implementation; if you're going to recognise something for a new feature, making existing keywords do dual duty is at least as much work as recognising a new keyword. $\endgroup$
    – Ben
    Jul 4, 2023 at 6:25
  • 8
    $\begingroup$ Never heard "soft keyword" before, I usually see "contextual keyword" instead... I can't guarantee it's more official, but at least it's more self-describing. $\endgroup$ Jul 5, 2023 at 12:29
  • 1
    $\begingroup$ PHP mostly gets around this by forcing variables to use the $ symbol, but there can still be conflicts for other things, such as in-built classes conflicting with user-defined classes. (I had to rename a String class we had in some of our older PHP websites.) $\endgroup$
    – TRiG
    Jul 5, 2023 at 17:41
  • 7
    $\begingroup$ @MatthieuM. for Python, "soft keyword" is definitely the official term $\endgroup$
    – muru
    Jul 6, 2023 at 7:05
  • $\begingroup$ The pure virtual function syntax isn't entirely bizarre. It can also be viewed as an abstraction leakage, as virtual functions are almost always implemented as pointers. $\endgroup$
    – Mark
    Jul 7, 2023 at 0:40
20
$\begingroup$

Reducing the number of reserved words

You normally can't name identifiers the same as keywords. That might be inconvenient. You may want a variable named class representing the HTML class attribute, or a function named catch.

Keywords are typically short verbs which are likely to conflict with functions or short variables.

You can have keywords that are not reserved by allowing them only in some situations. For example in Javascript in for (x of y) of is a keyword yet you can name your identifier of in other contexts.

More keywords should not make the compiler slower, in fact it is probably very slightly faster than aliasing keywords to multiple meanings that most languages use.

$\endgroup$
3
  • 1
    $\begingroup$ C++ is another language that has some context-dependent "keywords" - final, override and more only have their reserved meanings in particular places. $\endgroup$ Jul 5, 2023 at 8:34
  • 4
    $\begingroup$ @TobySpeight: Note that all of those were "added" to C++ after the first version, where any new keyword risked breaking backwards compatibility. $\endgroup$ Jul 5, 2023 at 12:31
  • $\begingroup$ Indeed, that was the mechanism to extend the language without adding new reserved words. $\endgroup$ Jul 5, 2023 at 13:40
14
$\begingroup$

TL;DR: No.

You raise a number of points, and a number of answers were given. I'll play Devil's Advocate and refute most/all of them.

Compiler Complexity and Performance

I suspect that more keywords may make implementing a compiler more complicated, slower as there are more keywords to compare tokens against or similar

On the contrary.

Compiler complexity is typically proportional to the number of concepts that the compiler has to deal with, and how much context the compiler needs to reconstruct (type inference, for example).

Using concepts with no keywords, or overloading keywords to mean multiple concepts in different contexts increases the amount of information the compiler must re-calculate (that the developer knew about) and therefore slows down compilation.

From a compiler perspective, one concept = one unique keyword is actually best.

Backward Compatibility

Or the inability to introduce a new keyword because it would prevent compiling older programs.

This is certainly an issue in many languages. Work-arounds include:

  • Using non-keywords: C++ uses = 0 for pure virtual functions, for example.
  • Using contextual keywords: C++ uses final and override as keywords only in function declarations.
  • Overloading existing keywords: C++ uses static in 4 different ways, with drastically different meanings.

And demonstrates an embarrassing lack of foresight.

A modern1 language should plan for evolution, and thereby figure out a solution to be able to introduce keywords without breaking backwards compatibility.

Examples:

  • Prefixing: if keywords start with :, and identifiers cannot, then there's no clash possible.
  • Versioning: in Rust, each edition introduces new keywords, and users opt-in to the new editions, fixing their code as they do2.

Thus, in a properly crafted language -- easier in hindsight -- there is no backward compatibility issue in adding keywords.

1: By which I mean a language created nowadays, with over 50 years of hindsight.

2: This is supported by another feature, which allow importing identifiers from previous versions, called Raw Identifiers. In essence, it applies the sigil strategy to identifiers (exceptionally), rather than keywords.

Reducing Reserved Words

There are many kinds of keywords.

The "traditional" keyword is reserved at all scopes, and there may be indeed a desire to keep that number low. There are, however, also contextual keywords, which are only keywords in certain contexts.

As an example, consider the (dubious) Rust test case:

fn union() {
    union union<'union> { union: &'union union<'union>, }
}

union is a keyword, when introducing an item, but otherwise can be used as function name, type name, lifetime name, and field name.

Other approaches to avoid "encroaching" on user variables exist, such as the unpopular option of prefix every keyword with a specific sigil3.

3 In this Rust example, single-quote is used as a sigil for lifetimes.

Simplicity and Learning Curve

The difficulty in learning a programming language is generally about semantics -- concepts -- and libraries -- standard or 3rd-party.

Having a single unique keyword for every concept is actually helpful for learners, as they make it easy to:

  1. Recognize concepts they have not learned yet.
  2. Search for them -- by keyword! -- on the web.

Keyword-less approaches, or keywords overloading, make a programming language less discoverable as it makes formulating web search queries more difficult, and reduces the pertinence of the search results.

Conclusion

One may want to reduce the number of concepts to keep a language simpler, both for users and compiler-writers.

Reducing the number of keywords without reducing the number of concepts, however, is foolish. If anything, it makes the language harder to learn.

On the other hand, one may want to use mostly contextual keywords, to avoid reserving too many desirable user-defined names.

And of course, one should plan for evolution and have a plan to introduce new keywords in the future. External versioning is a potential solution.

$\endgroup$
5
  • $\begingroup$ re: "plan for evolution" - I would love for js to have had this from the beginning instead of the mantra "one js". I understand there are very practical benefits to backward compatibility - and they potentially outweigh a versioned language. But darn it, removing baggage via convention and tooling (e.g. eslint) makes the language much more difficult to learn than if the baggage was removed from the language via a version update, making it eventually non-existent for newcomers. $\endgroup$
    – aaaaaa
    Jul 6, 2023 at 19:15
  • $\begingroup$ @aaaaaa - what would you call javascript's strict mode? It removes several features from the original language that are frequently the cause of mistakes and it's not merely a convention. $\endgroup$ Jul 7, 2023 at 5:38
  • $\begingroup$ you're right damien - they were wrong from the beginning. It's always been "two js". $\endgroup$
    – aaaaaa
    Jul 7, 2023 at 12:16
  • $\begingroup$ I would say contextual keywords are much harder for users of the language. It's very unclear to me what is going on in the union example, because I can't tell at a glance what is a keyword and what is not. The big advantage of contextual keywords in my mind is backwards compatibility. $\endgroup$ Jul 11, 2023 at 15:16
  • $\begingroup$ @CoffeeTableEspresso: The union example is taken to the extreme, and made worse by the fact that Rust has separate types and function namespaces: there is a single appearance of union as a keyword, in fact, so most of the confusion is fairly independent from it. $\endgroup$ Jul 11, 2023 at 16:09
11
$\begingroup$

You can simply use one of the lexerless parsing approaches (e.g., PEG), and then you would not need to worry about keywords. Your special meaning keywords will only be recognised as such when they make sense in a context, and you can still use the same words anywhere else (e.g., if or for as variable names, etc.).

$\endgroup$
2
  • 22
    $\begingroup$ The Fortran approach that allows such clear code as if if == then then then = else else else = if (and don't forget that the whitespace isn't required). $\endgroup$
    – Barmar
    Jul 4, 2023 at 14:20
  • 1
    $\begingroup$ @Barmar haha! Though to be fair, if you really want to write code as crazy as that you can do it in any language. Main::main main() {Main::main main = main;} is legal C++, if you first declare namespace Main {typedef int main;} $\endgroup$ Jul 6, 2023 at 15:09
7
$\begingroup$

I suspect that more keywords may make implementing a compiler more complicated, slower as there are more keywords to compare tokens against or similar

This is probably unfounded. There are two parts to it:

  • The lexer has to tell whether a token like while or count is a keyword or an identifier. This can be done by looking up the string in a hashtable, which takes O(1) time independently of the number of keywords.
  • When the parser sees a keyword token, it has to branch on the kind of keyword; for example, on an if token call parseIfStmt, on a while token call parseWhileStmt, on a throw token call parseThrowStmt and so on. This can be done in a switch statement which can be compiled to something more efficient than just checking the keywords one-by-one. If the lexer has already reduced keywords from strings to enum values, then a dense switch can use a jump table, whereas if they are still strings then a hashtable is a good option. Either way, the time taken to choose the right branch doesn't have to scale with the number of keywords.

So having more keywords probably has very little effect on the compiler's performance, in and of itself.

That said, having more concepts in the language might mean the compiler has more work to do after parsing, and if the additional concepts in the language correspond with additional keywords then these two things would be correlated.

$\endgroup$
6
  • $\begingroup$ If anything, I would argue that overloading a keyword makes the job of the subsequent passes more complicated, as the exact meaning now depends on the context which must be looked up... $\endgroup$ Jul 5, 2023 at 12:32
  • $\begingroup$ @MatthieuM. The context is generally already known by the parser, implicitly based on the control-flow. For example when parseExpr sees an if keyword it can only be an if expression, whereas if parseStmt sees the same keyword it can only be an if statement. $\endgroup$
    – kaya3
    Jul 5, 2023 at 12:38
  • 1
    $\begingroup$ In this specific case, yes. Ask C++ about using <> to delimit template parameter lists though... $\endgroup$ Jul 5, 2023 at 12:49
  • $\begingroup$ @MatthieuM. Well, if you will choose to have an ambiguous grammar then you're going to have problems when you have to parse it. That's not surprising, but many languages overload keywords without making their grammar ambiguous. $\endgroup$
    – kaya3
    Jul 5, 2023 at 12:50
  • $\begingroup$ Even if you do a linear search to lookup keywords, it won't really matter for performance. Most languages have dozens (or a few hundred in large cases) of keywords... $\endgroup$ Jul 11, 2023 at 15:18
6
$\begingroup$

Simplicity and Learning Curve

The less keywords and the more intuitive and related they are, the more simple and, as a such, easy to learn the language is. This is very useful especially in beginner languages.

$\endgroup$
4
  • 8
    $\begingroup$ Languages with few keywords typically compensate by just giving each keyword many meanings. This just makes the language more complex, not simpler $\endgroup$
    – mousetail
    Jul 4, 2023 at 9:59
  • 1
    $\begingroup$ True enough I was thinking more about having more complex things be just a few simple keywords. For example, in real languages, in spanish sheets are basically called "bed clothes". Thats more what I mean @mousetail $\endgroup$ Jul 4, 2023 at 11:11
  • 1
    $\begingroup$ Actually I agree on simplicity. Not necessarily for beginners. FP languages have significantly less keywords. FP experts can write very concise code with less keywords and can move between FP languages very quickly where as the same does not happen for experts in other languages. $\endgroup$ Jul 4, 2023 at 18:12
  • 1
    $\begingroup$ Also, replacing keywords with symbols reduces keyword count but not complexity: or vs | vs ||, begin vs {, and so on. $\endgroup$
    – Pablo H
    Jul 5, 2023 at 14:29
3
$\begingroup$

You can have Zero reserved keywords if you want, and this can be useful if you want users to define their own language extensions in yours.

Take a simple reserved keyword like the predefined type int: while it is a predefined type, it could belong in a specific package/module, the standard library. You could name it std.int and free the name int for other usages. User-defined modules could have their own int types.

And this can apply to keywords that are used to structure your source code, for example:

class MyClass {
};

fn f() {
};

The class or fn keywords are necessary here to know how to parse the following text. They are present at the syntactic level of a parser, but maybe you can avoid that and have a more regular parser and a single keyword declare:

declare(std.class)(MyClass){
}

declare(std.fn)(f, ()) {
}

Here, std.class and std.fn are identifiers that the compiler knows how to handle under the declare expression. They are not special anymore on a syntactic level, they are identifiers in a namespace for which the language specifies a behavior.

Each declaration has the same syntax, and it can be extended to include things like final, static, mutable, etc.

In the case of iterators like for, while etc. you can also imagine that they are standard std.for etc. identifers that the compiler knows how to interpret, all under the iterate keyword.

So now, we introduced both a declare keyword and an iterate generic keywords, but they are similar and could be merged as a single magic keyword:

magic(std.class)(MyClass) {
}

magic(std.for)(i, 0, 100) {
}

Your compiler can detect if the argument list of the std.for is malformed, the error detection is moved from the lexer/parser to one step further in the compiler where you check if syntactically well-formed expressions are meaningful.

And if you make all your syntax even more regular, you can avoid completely the magic keyword, which is how you end up with Lisp:

(cl:defclass my-class () ())

(cl:defun foo (a b c) ...)

(cl:loop ...)

All the predefined symbols that name types, functions, macros, special operators, etc. belong to the "COMMON-LISP" package (nicknamed cl), you can have your own loop, defun or defclass functions, macros or compiler-macros in your own package.

$\endgroup$
2
$\begingroup$

If you had many you can add more

If your language already had many keywords, you can add more.

For example you can define + operator in class:

def __add__(...):
    ...

This code is clear, but if your language had many keywords, you can add more:

public static int operator +(..., ...) {
    ...
}
$\endgroup$
1
$\begingroup$

Conciseness

Keeping the number of keywords to a minimum reduces the chances that the developer will need to pick longer, more descriptive names. This means that the language will also be more concise and readable, in most cases.

As well as making the language more concise, reducing the number of keywords prompts the designer to pick more descriptive words, as keywords must be used sparingly.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .