7
$\begingroup$

I recognize that not everyone designing a new language is doing it to compete with other languages in the same problem/application-domain and achieve (and retain over time) similar adoption levels. But for those who are, what overarching, high-level design strategies matter and why? How do those design strategies help achieve that goal?

Some additional qualification / scope narrowing of the question:

  • I realize that the words "long-term adoption" and "widespread adoption" are pretty murky/ambiguous. More concrete descriptions might might be "adoption growing and lasting over the next 20+ years", and "adoption at a similar scale to other widely-adopted languages occupying a similar problem-domain(s)".

  • This is a question about overarching design strategies for the stated goal- not about things like the language design team having enough funding, having good teamwork, quality of "product", rollout, or investment in marketing. It's about what the language has to do or be to have a place in the ecosystem it wants to enter- not about what the people designing it have to do on top of designing it.

  • This question applies equally to new languages (and their tooling ecosystems) designed to compete with other ones in a similar problem/application-domain, or new languages designed to evolve such existing languages in ways that the designers of those languages choose not to.

Please give concrete examples from languages who have used the design strategies you name and have went on to achieve the stated goal. I.e. back up your claims with evidence. This question intentionally doesn't constrain the sought examples to any one specific language. What I'm seeking here is a bigger-picture view.

Another title for this question might have been (sorry for any leading of the question here): "What overarching design strategies have contributed to the adoption of languages such as C++, TypeScript, Kotlin, and others?" (because I think they're mention-worthy examples of languages that have (to varying but significant degrees) succeeded in attaining those goals in those circumstances)

$\endgroup$
2

2 Answers 2

8
$\begingroup$

Attribution: This answer presents the ideas of Herb Sutter, based on his talk at cpponsea 2020, "How to answer "why will yours succeed, when X, Y, and Z have failed?"". I've tried to elaborate by adding some additional examples.

The overarching strategies Herb presents are value, availability, and compatibility (with focus on compatibility), where the points made about value and availability are deeply connected to the point about compatibility.

These points are presented from an analysis of common design strategies of the languages that came into problem/application-domains with pre-existing, well-established languages, and which survived, going on to achieve usage-levels of comparable scale to the languages which came before them.

Strategy 1: Value by-Design

This strategy is about answering the question "why would anyone want to go from the language they're using to this new language?"

To do that, you need to make a distinct, articulable new value proposition (like making an elevator pitch), and the easiest way to do that is to address existing/known pain points with already-adopted languages that already resonate with people and are well-understood- not just solving the same problems in the shared problem/application-domain, but also problems in existing solutions. For example (note: I find some statements in these "elevator pitches" subjective, but have left them as-is intentionally):

  • From the ISO C++ wiki: C++ is C with stronger type-checking (addressing pain-points with C's type system), wider support for different programming styles (but still including those of C, while allowing you to program "closer to the problem domain than the solution domain"), and more notational support without loss of efficiency (this isn't a pain-point for all C users, but it is to some), and support for data abstraction, object-oriented-programming, resource management, and generic programming (addressing pain-points related to a (relatively speaking) lack of those features in C). That's in terms of features. In terms of the problem/application-domain, from the ISO C++ wiki's section on why C++ was invented states: "Stroustrup wanted to write efficient systems programs in the styles encouraged by Simula67. [...] The more general aim was to design a language in which developers could write programs that were both efficient and elegant. Many languages force you to choose between those two alternatives."

  • From TypeScript's website: "TypeScript is JavaScript with syntax for types." (you can guess what the pain-point is) . See also the design goals, which include: "Statically identify constructs that are likely to be errors." and "Provide a structuring mechanism for larger pieces of code."

  • From Kotlin's Getting Started docs: "Kotlin is a modern but already mature programming language aimed to make developers happier. It's concise, safe, interoperable with Java and other languages, and provides many ways to reuse code between multiple platforms for productive programming." (Ex. pain-points with concision in Java)

There are at least two pitfalls with designing value propositions for a new language:

  • To underestimate the value of of existing languages and code written in them. What a designer of a new language might call "legacy / old / complex" in an existing language is what a programmer in that language might as equally likely call "proven / working / robust (with flaws)"
  • To underestimate the cost of parity: having already built something in one language, and then having to spend what it takes to build that again in a new language while achieving similar feature, performance, and stability (and often having to continue to support the previous version).

Strategy 2: Availability by-Design

This strategy is about answering the question "how do I make the value of this new language available to users of existing languages in this space?"

The "Holy Grail" of this aspect of strategy is to be able to achieve a solution where users of well-established languages in the same problem/application-domain can easily add code in your new language to their existing projects written in the existing languages.

This can be seen in the following technologies (and bleeds into the next section's point on the strategy of compatibility):

  • From the ISO C++ wiki's section: "Is C a subset of C++?": "C++ supports every programming technique supported by C95 (C90 plus an Amendment) and earlier. Every such C program can be written in essentially the same way in C++ with the same run-time and space efficiency. It is not uncommon to be able to convert tens of thousands of lines of ANSI C to C-style C++ in a few hours." See also the ISO C++ wiki's FAQ, "Is C++ backward compatible with ANSI/ISO C?" (TL;DR "Almost")

  • Cfront (tooling): A compiler that compiled C++ to C. It was therefore available for use with (compatible with) with any C tooling.

  • TypeScript: You can see availability written into the design goals: "Preserve runtime behavior of all JavaScript code.", "Use a consistent, fully erasable, structural type system.", and "Be a cross-platform development tool."

  • Some Kotlin-related tools have features to translate Java code into Kotlin code (Ex. JetBrains IDEs). Kotlin can be compiled to Java bytecode (that's not particularly interesting, since a lot of other languages do too, but I find it noteworthy here).

  • (tooling): Swift is available to developers through Xcode on any Objective-C-supported platform.

  • (tooling): Rosyln C# compiler is available to developers through Visual Studio on Visual C#-supported platforms.

Strategy 3: Compatibility by-Design

Look at an adoption function as relating value as a function of effort. Would you as a user prefer an adoption function like a single-step step function (where all of the effort needs to happen all at once and you go from zero to 100% of the value), or an adoption function like a linear function (where you can put in the effort gradually and reap the value in proportion to the effort you have put in so far)?

Herb presents a minimum bar that code written in the new language should be able to seamlessly use code from the existing language (/languages)- whether through source or binary compatibility (Ex. FFI, byptecode, etc.). Ideally it would be seamless both ways, but old-to-new should be seamless at a minimum.

The "Holy Grail" of this design strategy would be that it take zero-effort to go from the existing language to the new language.

The examples given in the "Availability by Design" section also make sense here. In addition:

  • Any C++ code can call C code (see also the ISO C++ wiki's page on "How to mix C and C++"). In the 1980's (no longer true for newer C++ language versions, but still "largely true"), you could give C source code to a C++ compiler and it would compile and give the same behaviour.

    Quoting from a review of D&E by Al Stevens written for Dr. Dobb's Journal, August 1994 (you can find a snippet of it here):

    [Stroustrup] could have assigned less importance to compatibility with C. "Within C++, there is a much smaller and cleaner language struggling to get out," which he says "would ... have been an unimportant cult language." Second, he is committed to the concept of static (as opposed to dynamic) type checking as being inherently safer and essential to retain the efficiency of C. Without that guarantee, programmers used to C's efficiency will not switch to a new language no matter what promise it holds.

  • Any valid JavaScript code is valid TypeScript code (and you can already start getting some of the benefit of the TypeScript compiler's static analysis with almost no effort). Also in TypeScript's design goals: "Impose no runtime overhead on emitted programs.", "Emit clean, idiomatic, recognizable JavaScript code.", "Align with current and future ECMAScript proposals." (a degree of forward compatibility with future JS), "Do not cause substantial breaking changes from TypeScript 1.0." (a degree of self-backward compatibility between major versions).

  • From SASS's "Basics" documentation page: "Sass has two syntaxes! The SCSS syntax (.scss) is used most commonly. It's a superset of CSS, which means all valid CSS is also valid SCSS."

  • Swift has very good bi-directional compatibility with Objective-C. They can call into each other, Objective-C object-layout and reference-counted-lifetime models are the same in Swift, and tooling support can do bridging header generation and view Object-C code as equivalent Swift code.

In those examples, compatibility came at costs- whether in implementation or in costs of holding on to some problems from existing languages, but it was deemed worthwhile to achieve the desired levels of adoption.

Some "counterexamples":

  • Dart didn't achieve high compatibility with JS from the start, and it's still working on it in at the time of this writing: https://github.com/dart-lang/sdk/issues/35084.

  • Borland Delphi did very well with designing for value and availability, and achieved compatibility at the start, but couldn't maintain it (difficult due to Borland being a separate company from Microsoft, which led to Borland always being behind the latest platform features).

  • Python 3 is source incompatible with Python 2, and requires a mix of tool-assisted and manual migration (see also https://docs.python.org/3/howto/pyporting.html). And even nine years after Python 3 was released in 2008, a study showed that a significant amount of (purportedly most) code was still using a mix of Python 2 and 3 (source), and even though Python 2 was officially frozen and unsupported in 2020, the JetBrains developer survey of October 2019 found that only 90% of respondents were using Python 3.

  • C++11's ABI break banning reference-counting for std::string. GCC implemented the conforming implementation in 2015 with version 5.1, and adoption of that took years. The feature was disabled by default on Red Hat Linux's GCC 8 until 2019.

There's a pattern in the above examples: historically, it has often taken ~10 years to adopt new things without a strong compatibility bridge. The lesson being- if you want to do something that isn't compatible with well-established solutions in the problem/application-space and you have a good value proposition, don't be surprised if adoption to "well-established" levels takes something on the order of 10 years / an extra 10 years.

Some other compatibility-related pitfalls given the stated goals include (at 41:39): function "colouring" (dialect bifurcation) and the same code meaning different things to different but largely languages or language versions.


Another interesting note (but probably better left to another Q&A): these newer languages sometimes influence the existing languages:

$\endgroup$
2
  • $\begingroup$ feel free to suggest additional examples either by suggesting edits or by commenting. Either way, please provide evidence to back up the info (Ex. links to more or less "official" (from the mouths of the maintainers) writings as I have done). $\endgroup$
    – starball
    May 19, 2023 at 21:30
  • $\begingroup$ Kotlin is notable for having very high interoperability with Java (unlike, for example, Scala), having made it a design point. Java code is easily usable from Kotlin and the Kotlin team tries to make sure you can use as much Kotlin code from Java as possible. I often have projects where Java and Kotlin code are mixed in the same project. $\endgroup$
    – Seggan
    Nov 5, 2023 at 15:22
2
$\begingroup$

When developing a new language, existing languages which have built up stability and community will always have the advantage. So you really need a way to make your language worthwhile.

Here is what Rust, Kotlin, TypeScript, and Swift (all languages which have superseded others in their domain) have done.

Significantly improve upon the competition

You can't just fix one small problem and expect others to prefer your language. Rust's greatest strengths are C++'s greatest weaknesses: easy package management, safe memory-management, sensible generics, actual "Algebraic Data Types". Also, any of these features alone wouldn't have given Rust the widespread adoption it has today: a C++-like with strict safety guarantees would probably get adopted by safety-critical, performance-critical programs, but it would never reach mainstream adoption or adoption anywhere near C++'s numbers. Swift also out-done Objective-C with better performance and modern language design, and Kotlin out-done Java with less verbosity and footguns; but even these languages needed more than improvements alone to gain popularity...

Industry support

Rust was developed with the help of Mozilla, Swift was developed by Apple, Kotlin was developed by JetBrains, and TypeScript was developed by Microsoft. A random nobody, or even a super-talented developer whose famous but only in the inner tech circles, will have a much harder time building up their language's community. And community is what makes or breaks a language, as technology keeps evolving: you need community to provide the utility libraries, resources, and platform support necessary for your language to become usable (and therefore get even more community). Industries also have the ability to make their language the default in their other products and platforms: Swift was created to be the main language for macOS and iOS, and Kotlin only really became popular once it became the main language for Android.

Integration with existing tooling

This is where TypeScript in particular shines: not only can TypeScript code import JavaScript code and vice versa, not only does it compile to JavaScript, but TypeScript compiles to readable JavaScript and thus can even be used by JavaScript debuggers (when sourcemap fails, but this happens suprisingly often). Swift has interop with Objective-C, Kotlin has interop with Java and can target every JVM, and Rust has interop with C++.

It takes time and stabilization

None of these languages were popular when officially released, and even today, there are many projects which would clearly benefit from them but have yet to migrate. Before it becomes popular, a language needs to stabilize: nobody wants to use a new language only to have to throw out their code because of massive API-breaking changes (which often happened in before Rust 1.0 and Swift 3, or was it 4?) A language also needs to become reliable: nobody wants to use a buggy language, and even with ample testing and good practices a new language is practically guaranteed to have many bugs (even Rust and Kotlin still have some bugs today). And even if the language itself is perfect, it needs community resources and packages, developer training, and a general sign that it and its community is here to stay. Ultimately, many of these languages took years of development before they became remotely mainstream; developing a new language is a long-term effort.

$\endgroup$
3
  • $\begingroup$ hm. While important, I'm not sure if "Industry support" is really a design strategy. See also the question post's constraints, which states: "This is a question about overarching strategies for the stated goal- not about things like the language design team having enough funding, having good teamwork, quality of "product", rollout, or investment in marketing." $\endgroup$
    – starball
    May 20, 2023 at 0:52
  • $\begingroup$ note: the rest of what you've written largely seems to be related to or repeating Herb's points in his cpponsea talk (which I've used in my answer post). $\endgroup$
    – starball
    May 20, 2023 at 0:54
  • $\begingroup$ to elaborate, your "Significantly improve upon the competition" sectionreally only adds the rust example to Herb's point about "value by-design", your "Integration with existing tooling" section is just a subset of Herb's points (and my extended examples) about availability by-design and compatibility by-design, and your "It takes time and stabilization" section is strongly related (a subset of?) Herb's side-point on the consequences of not having a good compatibility by-design strategy. $\endgroup$
    – starball
    May 20, 2023 at 1:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .