2
$\begingroup$

Is the mass adoption of Unicode tokens as operators in general-purpose programming languages ​​a good idea? How acceptable is such a language to ordinary users and developers?


Background

I want to add a lot of Unicode brackets and operators to my language, for use as literals for various commonly used structures and quote-free paths, because I think Unicode tokens are more visually noticeable than ASCII combination operators, with or without highlighting support.

I did some work to address issues I saw, hoping to improve the workflow:

  • Designed a set of Unicode-based literal tokens
  • Adjusted the lexer to support pure QWERTY input
    • Unlike Julia, this is not a tab-completion, but a literal
    • Mainly supports unicode tokens by combining (::() / .()) and escaping (\X).
  • Implemented a formatter for Jetbrains IDEs, which can automatically replace ASCII tokens with their Unicode counterparts during formatting

According to the existing design, it looks roughly like this

Collection QWERTY Compatible Unicode Token
Compare a < b > c (a < b) > c
Generic A::<T> A⟨T⟩
Block { STATEMENT }
HashMap #{ID: EXPR} ⦃ID: EXPR⦄
List [EXPR]
HashSet #[EXPR] ⟦EXPR⟧
Range @[START:END:STEP] ⁅START:END:STEP⁆
Function Invoke ID(ARG)
Function Builder(like swift) TYPE::(AST) TYPE⦅AST⦆
Label \lID: LOOP ¶ID: LOOP
Localized \Lgroup.key(SLOT) §group.key(SLOT)
Comment \\ COMMENT ⍝ COMMENT
Nil \N

(There are additionally also some prefix, infix and postfix operators which are not listed.)

Are there any other issues with implementing a language so heavily reliant on Unicode?

$\endgroup$
9
  • 2
    $\begingroup$ I'm voting to close this question because it doesn't provide any objective way to determine what a good answer is. If your question includes phrases like "Does such language make you feel bad?", it is probably not a good fit for this site. $\endgroup$
    – Ginger
    Commented Apr 18 at 14:21
  • 2
    $\begingroup$ Obviously opinionated, but please take readability into account. At least for me, the symbols in Range, HashSet and HashMap looked more or less the same until I gave it a deeper look. The same happened with the Unicode equivalent for Compare and Generic. It could very well lead to developers replacing only the closing character, not realizing it's not a regular character and spending hours confused. $\endgroup$
    – kouta-kun
    Commented Apr 18 at 16:28
  • 1
    $\begingroup$ Keep in mind that supporting Unicode is a nontrivial security issue. See UTR #36. $\endgroup$
    – RobertR
    Commented Apr 18 at 18:17
  • $\begingroup$ As well as needing some objective criteria, this question needs a bit of focus. Is it about using non-ASCII characters to expand the symbology of the language - which has been explored since before Unicode, e.g. in APL? Is it about using visually similar but distinct characters, as is the case with the different types of bracket? Is it about having one form for input and a translation tool, which would explain the otherwise irrelevant example of inserted parentheses in the compare syntax? Is it about encoding (Unicode) or text entry (QWERTY) or even display (compare: programmer's ligatures)? $\endgroup$
    – IMSoP
    Commented Apr 18 at 19:30
  • 3
    $\begingroup$ I think the main objection most would have to this is simply that it's inconvenient to type these things, since they don't exist on keyboards. You can use something that translates from the QWERTY syntax to Unicode, but then what's the point? Is it just because the Unicode version is "prettier"? But it means we need to know two syntaxes: One for typing, another for reading. $\endgroup$
    – Barmar
    Commented Apr 19 at 23:05

1 Answer 1

2
$\begingroup$

You can read about the (defunct) Fortress programming language. They implemented Unicode mathematical operators galore.

Check out one of the post-mortems: Fortress Features and Lessons Learned.

Also, search for "The Fortress Language Specification", and papers from Guy L. Steele Jr., for example.

Update (for summary request): Of course a summary would be nice, but I'm not an expert (by far), just enthusiast. Let's try adding something, at least.

Some quotes from "Fortress Features and Lessons Learned":

Syntax strongly influenced by mathematics and whiteboards
– Fortress libraries encourage use of ∪, ∩, ⊆, ∈ for sets
– Also |a| , b x c , d x e , p ∧ q , p ∨ q , ¬q , p ⊕ q
– Multiplication expressed as a · b , a × b , or just a b

Fortress was a project which competed with others to be "the new Fortran", a programming language for High Performance Computing (it was not selected, though). So I guess being able to replicate mathematical language was intended to appeal to scientists and applications for scientific computing.

One point of note is that, to replicate or mathematical notation, they added support for "whitespace operator", juxtaposition. They say it was possible but complicated things. I think it did not end up well, perhaps.

Adding lots of operators had some other effects: operator precedence, and typesetting.

Nontransitive Operator Precedence in Fortress • It’s okay to write a + b > c + d
– Why? Because + has higher precedence than >.
• And it’s okay to write p < 0 ∨ p > 9
– Why? Because > has higher precedence than ∨.
• But in Fortress it is not okay to write a + b ∨ c + d.
– There is no precedence defined between + and ∨.
• We use only the most obvious and familiar rules of precedence.
– This reduces opportunities for for making silly mistakes.
• This matters when you have hundreds of operator symbols such as ∪ ∩ @ ÷ ⊕ ⊗ ∈ ≈. [this do not print well: 4 f } ? b l k ]
• Not a big burden—simply use parentheses to make meaning clear:
(a + b) ∨ (c + d) or a + (b ∨ c) + d

I have the impression that implementing typesetting required substantial effort. And gives you two versions of code: typeset vs ASCII. But perhaps typesetting was mostly for "big operators" (sum, lim, max) as in LaTex?

Another Point about Syntax
• There are tradeoffs between readability and writability
• Mathematical syntax was designed more for certain kinds of concision than for robustness!
– In particular, simple juxtaposition is used in many ways
– We tried to make this programmable
-- Sometimes this worked beautifully
-- Sometimes it required some weird contortions
• Haskell and Scala do this more consistently
– Each captures a certain PART of mathematical tradition quite well

So several features interact with others. And design decisions were made as a whole, so to extract guidance useful to you you will have to look into the details. Or ask the authors themselves. :-)

$\endgroup$
1
  • 2
    $\begingroup$ Can you share a summary of the links you provided - did they decide it was a good thing or not? What points do you specifically think are relevant to the question? Not only does this save everyone reading a lot of time, it means the answer would remain relevant even if the links stop working in future (which is, sadly, extremely common). $\endgroup$
    – IMSoP
    Commented Apr 18 at 17:05

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .