Attribution: This answer presents the ideas of Herb Sutter, based on his talk at cpponsea 2020, "How to answer "why will yours succeed, when X, Y, and Z have failed?"". I've tried to elaborate by adding some additional examples.
The overarching strategies Herb presents are value, availability, and compatibility (with focus on compatibility), where the points made about value and availability are deeply connected to the point about compatibility.
These points are presented from an analysis of common design strategies of the languages that came into problem/application-domains with pre-existing, well-established languages, and which survived, going on to achieve usage-levels of comparable scale to the languages which came before them.
Strategy 1: Value by-Design
This strategy is about answering the question "why would anyone want to go from the language they're using to this new language?"
To do that, you need to make a distinct, articulable new value proposition (like making an elevator pitch), and the easiest way to do that is to address existing/known pain points with already-adopted languages that already resonate with people and are well-understood- not just solving the same problems in the shared problem/application-domain, but also problems in existing solutions. For example (note: I find some statements in these "elevator pitches" subjective, but have left them as-is intentionally):
From the ISO C++ wiki: C++ is C with stronger type-checking (addressing pain-points with C's type system), wider support for different programming styles (but still including those of C, while allowing you to program "closer to the problem domain than the solution domain"), and more notational support without loss of efficiency (this isn't a pain-point for all C users, but it is to some), and support for data abstraction, object-oriented-programming, resource management, and generic programming (addressing pain-points related to a (relatively speaking) lack of those features in C). That's in terms of features. In terms of the problem/application-domain, from the ISO C++ wiki's section on why C++ was invented states: "Stroustrup wanted to write efficient systems programs in the styles encouraged by Simula67. [...] The more general aim was to design a language in which developers could write programs that were both efficient and elegant. Many languages force you to choose between those two alternatives."
From TypeScript's website: "TypeScript is JavaScript with syntax for types." (you can guess what the pain-point is) . See also the design goals, which include: "Statically identify constructs that are likely to be errors." and "Provide a structuring mechanism for larger pieces of code."
From Kotlin's Getting Started docs: "Kotlin is a modern but already mature programming language aimed to make developers happier. It's concise, safe, interoperable with Java and other languages, and provides many ways to reuse code between multiple platforms for productive programming." (Ex. pain-points with concision in Java)
There are at least two pitfalls with designing value propositions for a new language:
- To underestimate the value of of existing languages and code written in them. What a designer of a new language might call "legacy / old / complex" in an existing language is what a programmer in that language might as equally likely call "proven / working / robust (with flaws)"
- To underestimate the cost of parity: having already built something in one language, and then having to spend what it takes to build that again in a new language while achieving similar feature, performance, and stability (and often having to continue to support the previous version).
Strategy 2: Availability by-Design
This strategy is about answering the question "how do I make the value of this new language available to users of existing languages in this space?"
The "Holy Grail" of this aspect of strategy is to be able to achieve a solution where users of well-established languages in the same problem/application-domain can easily add code in your new language to their existing projects written in the existing languages.
This can be seen in the following technologies (and bleeds into the next section's point on the strategy of compatibility):
From the ISO C++ wiki's section: "Is C a subset of C++?": "C++ supports every programming technique supported by C95 (C90 plus an Amendment) and earlier. Every such C program can be written in essentially the same way in C++ with the same run-time and space efficiency. It is not uncommon to be able to convert tens of thousands of lines of ANSI C to C-style C++ in a few hours." See also the ISO C++ wiki's FAQ, "Is C++ backward compatible with ANSI/ISO C?" (TL;DR "Almost")
Cfront (tooling): A compiler that compiled C++ to C. It was therefore available for use with (compatible with) with any C tooling.
TypeScript: You can see availability written into the design goals: "Preserve runtime behavior of all JavaScript code.", "Use a consistent, fully erasable, structural type system.", and "Be a cross-platform development tool."
Some Kotlin-related tools have features to translate Java code into Kotlin code (Ex. JetBrains IDEs). Kotlin can be compiled to Java bytecode (that's not particularly interesting, since a lot of other languages do too, but I find it noteworthy here).
(tooling): Swift is available to developers through Xcode on any Objective-C-supported platform.
(tooling): Rosyln C# compiler is available to developers through Visual Studio on Visual C#-supported platforms.
Strategy 3: Compatibility by-Design
Look at an adoption function as relating value as a function of effort. Would you as a user prefer an adoption function like a single-step step function (where all of the effort needs to happen all at once and you go from zero to 100% of the value), or an adoption function like a linear function (where you can put in the effort gradually and reap the value in proportion to the effort you have put in so far)?
Herb presents a minimum bar that code written in the new language should be able to seamlessly use code from the existing language (/languages)- whether through source or binary compatibility (Ex. FFI, byptecode, etc.). Ideally it would be seamless both ways, but old-to-new should be seamless at a minimum.
The "Holy Grail" of this design strategy would be that it take zero-effort to go from the existing language to the new language.
The examples given in the "Availability by Design" section also make sense here. In addition:
Any C++ code can call C code (see also the ISO C++ wiki's page on "How to mix C and C++"). In the 1980's (no longer true for newer C++ language versions, but still "largely true"), you could give C source code to a C++ compiler and it would compile and give the same behaviour.
Quoting from a review of D&E by Al Stevens written for Dr. Dobb's Journal, August 1994 (you can find a snippet of it here):
[Stroustrup] could have assigned less importance to compatibility with C. "Within C++, there is a much smaller and cleaner language struggling to get out," which he says "would ... have been an unimportant cult language." Second, he is committed to the concept of static (as opposed to dynamic) type checking as being inherently safer and essential to retain the efficiency of C. Without that guarantee, programmers used to C's efficiency will not switch to a new language no matter what promise it holds.
Any valid JavaScript code is valid TypeScript code (and you can already start getting some of the benefit of the TypeScript compiler's static analysis with almost no effort). Also in TypeScript's design goals: "Impose no runtime overhead on emitted programs.", "Emit clean, idiomatic, recognizable JavaScript code.", "Align with current and future ECMAScript proposals." (a degree of forward compatibility with future JS), "Do not cause substantial breaking changes from TypeScript 1.0." (a degree of self-backward compatibility between major versions).
From SASS's "Basics" documentation page: "Sass has two syntaxes! The SCSS syntax (.scss) is used most commonly. It's a superset of CSS, which means all valid CSS is also valid SCSS."
Swift has very good bi-directional compatibility with Objective-C. They can call into each other, Objective-C object-layout and reference-counted-lifetime models are the same in Swift, and tooling support can do bridging header generation and view Object-C code as equivalent Swift code.
In those examples, compatibility came at costs- whether in implementation or in costs of holding on to some problems from existing languages, but it was deemed worthwhile to achieve the desired levels of adoption.
Some "counterexamples":
Dart didn't achieve high compatibility with JS from the start, and it's still working on it in at the time of this writing: https://github.com/dart-lang/sdk/issues/35084.
Borland Delphi did very well with designing for value and availability, and achieved compatibility at the start, but couldn't maintain it (difficult due to Borland being a separate company from Microsoft, which led to Borland always being behind the latest platform features).
Python 3 is source incompatible with Python 2, and requires a mix of tool-assisted and manual migration (see also https://docs.python.org/3/howto/pyporting.html). And even nine years after Python 3 was released in 2008, a study showed that a significant amount of (purportedly most) code was still using a mix of Python 2 and 3 (source), and even though Python 2 was officially frozen and unsupported in 2020, the JetBrains developer survey of October 2019 found that only 90% of respondents were using Python 3.
C++11's ABI break banning reference-counting for std::string. GCC implemented the conforming implementation in 2015 with version 5.1, and adoption of that took years. The feature was disabled by default on Red Hat Linux's GCC 8 until 2019.
There's a pattern in the above examples: historically, it has often taken ~10 years to adopt new things without a strong compatibility bridge. The lesson being- if you want to do something that isn't compatible with well-established solutions in the problem/application-space and you have a good value proposition, don't be surprised if adoption to "well-established" levels takes something on the order of 10 years / an extra 10 years.
Some other compatibility-related pitfalls given the stated goals include (at 41:39): function "colouring" (dialect bifurcation) and the same code meaning different things to different but largely languages or language versions.
Another interesting note (but probably better left to another Q&A): these newer languages sometimes influence the existing languages:
