Chapter 1 of "The Rust Programming Language" (Klabnik and Nichols) says:
[S]ome common Rust packages depend on C code and will need a C compiler.
Why do Rust packages have any dependency on C code? That is, why does Rust code ever need to use something that isn't also written in Rust?
Why do Rust packages have any dependency on C code?
Taken literally, this question is not really on-topic: it’s a question about some decisions some people made when they wrote some libraries, not a question about the Rust programming language. By and large, their motivations likely have more to do with the practicalities of software engineering than anything fundamental about programming languages. Ultimately, it’s just plain convenient to reuse C code because there’s a lot of it, it’s particularly easy to call into, C toolchains are ubiquitous, and whatever runtime support it needs is almost certainly already loaded into your process.
Still, if we follow that reasoning just a little bit further, there are definitely a few programming language questions hidden in there. For example, why is C so exceptionally easy to call from other languages? Why does Rust have built-in support for C calling conventions and struct layout but not other languages? This is by no means specific to Rust: in most languages, “foreign function interface” might as well be a synonym for “a way to call into C”, and it is not at all uncommon for C toolchains to show up in unexpected places.
Why is that?
The phrase “the C ABI” is something of a misnomer, as C does not specify an ABI, nor is it really defined at the level that would allow specifying one. Rather, “the C ABI” refers to a general collection of things defined for every combination of architecture and operating system. For example:
The calling convention specifies a standard set of rules for how arguments are passed to and returned from functions, the layout of the stack, and what registers are preserved across the call.
Datatype size and alignment rules specify the necessary size and alignment of primitive types like int, long, and float.
Structure type layout rules specify how composite structures are laid out in memory and how they are passed to and returned from functions.
Executable formats define how executables and shared libraries should be laid out so that the operating system knows how to load them into memory, set up their virtual address space, and dynamically link them.
It is easy to quibble about what is and is not part of “the C ABI” for a given platform. Are executable formats really a part of the C ABI, or are they something else? What about thread local variables? Signal handling? Debugging symbols and unwinding information? I’m not going to try to pin down some precise list because there simply isn’t one. What matters is two things:
Where precisely the C ABI and C standard library end and your operating system begins is not at all clear.
It is extraordinarily important that the C ABI for a given platform does not ever change.
These two things are very closely related.
As apropos’ answer points out, there is a real sense in which C isn’t (just) a programming language. A staggering amount of essential operating system functionality is not just most convenient to access via C libraries, it’s the only supported way of accessing it. Sure, if you really want, you can find the conventions for making a syscall “directly”, but the idea that syscalls are the interface between userland and the operating system is rather idealistic. For one, syscalls are the interface between userland and the kernel, and the kernel is really an exceedingly small part of even a fairly minimal modern operating system. For another, they are poorly documented and require reverse-engineering on proprietary operating systems, and worse, syscalls are not even stable on Windows.
So it is really rather black and white: the intended way to call into your operating system is to call a C function. In fact, let me make this even more explicit: it is not just the intended way for a human to write a computer program that calls into your operating system, it is the intended mechanism by which every programming language in existence calls into your operating system. On Linux, one can perhaps quibble with this (though I still think it would be quite ill-advised), but on macOS and Windows, you really cannot, so there’s just no way around it—your programming language needs some way to (eventually) make C calls.
That said, if your programming language is interpreted, you basically don’t have to worry about this problem. Whatever language your interpreter is written in already knows how to make the necessary calls, so you can just defer to that. In fact, even a compiled language that uses its own binary object format and does its own linking and loading and has its own runtime can largely get away with not knowing how to generate C calls because the program can always yield to the runtime’s scheduler, which is written in a language that knows how to perform the call into C.
This means that, counterintuitively, being a systems language without a runtime (like Rust) means you actually have to care more about the details of the C ABI on each and every platform because you have to be able to generate code that makes those calls directly. You have to speak the platform’s calling convention, you have to do all your stack layout in a way that’s compatible with the C stack, you need mechanisms for generating code that uses the correct structure layout and alignment rules, and you need to figure out what to do when a function written in your language returns to a function written in C, something that is not altogether obvious when your language provides functionality like exceptions that is not supported by the C ABI.
After everything I said above, you hopefully understand why just about every programming language under the sun needs some way to speak the platform’s C ABI, but you might wonder whether calling it “the C ABI” is very useful. You might think it would be clearer if we called it “the platform ABI,” and in fact that is often done! This might lead you to believe that we could dispense with all the programming language parts of C—like type declarations and header files and .o objects—and just focus on the binary interface bits, which really don’t need to be considered through the lens of C at all.
Sadly, if you actually try to implement a compiler, you will rapidly discover that this ABI is painfully difficult to disentangle from the programming language C. In order to make a C call, you don’t just need to know what registers to pass arguments in and what size an int is, you need to know things like struct layout and what it means to spill a struct onto the stack. You also need some way to take the interface specifications provided by operating systems and turn them into a format your compiler can work with, and wouldn’t you know it, those interface specifications are provided in the form of… C header files.
No, really: go down this rabbit hole and it will start to drive you mad. Pick just about any syscall that involves a struct type and try to figure out how to automatically determine how to generate the right code to use it. Consider the case of stat. The man page says it’s in sys/stat.h, but how do we even find where that header is on some arbitrary system? The highest-voted answer to a Stack Overflow question about how to do this on Linux is to call gcc. I assure you it does not get any easier to avoid giving in and calling a C compiler from here.
The reality is that C header files are currently used as the lowest-level interface description language you can get your hands on, and this is a somewhat frustrating choice seeing as it is virtually impossible to parse them without writing a C compiler. But let’s suppose you did that—you deduced all the platform-specific rules for layout and calling conventions, you reimplemented the C preprocessor, and you wrote a fully-featured C parser and symbol resolver that allows you to parse any standard C header file—you will rapidly run into another problem.
The header files on your system are not written in standard C.
This part can come as a bit of a shock to people, especially since there’s some idea that the header files found on your computer are compiler-agnostic. In fact, if you go digging through your system’s header files, you’ll find an awful lot of #ifdef __GNU_C__ and #ifdef __clang_major__ littered about. In practice, your header files are very much vendor-specific, they just explicitly handle a small list of known vendors.
At this point it may become clear how hopeless this all is. The deeper you look, the fuzzier the line between your operating system and the C compiler becomes. In a way, the entire C toolchain is a part of the operating system, the wildly complex decoder ring you are forced to use if you want to be able to conform to the eye-wateringly elaborate machinations required to call into your operating system. Yes, I realize this might sound hysterical, but consider just how deeply the C toolchain is interwoven with languages that, frankly, would rather have as little to do with it as possible:
Rust compiles to LLVM, and rust-bindgen uses libclang.
GHC tries its best to implement a common subset of the platform calling convention for its FFI, but users found this insufficient enough that it now supports a mode that just generates a .c source file containing a stub function and compiles it with a C compiler.
The Swift compiler straight-up includes an embedded libclang.
There will always be people who quixotically struggle to avoid depending on the C toolchain at all costs, but doing so is essentially hopeless. Even if one were to implement everything required to consume system header files without MSVC, GCC, or Clang, support for those headers is such a moving target that it would likely create a headache-inducing maintenance burden. What’s more, C’s exceptional ubiquity only entrenches it further: when software written in different languages needs to coexist in the same process, the mediator is the collection of languages, tools, runtime infrastructure, and operating system support that we now know under the moniker “C”. Why not something else? Because everything already supports C. So everything has to support C.
To quote the current Twitter bio of JeanHeyd Meneide, current Project Editor for the C standardization committee:
The C Standard Cannot Be Replaced And Will Never Be Destroyed.
Quite.
cdecl vs stdcall. C mostly came into the picture with the dominance of UNIX.
$\endgroup$
Commented
Nov 27, 2023 at 0:29
(note: this answer is addressing a quite different question from the original revision: why one language would have a dependency on another language?)
Is there is something fundamental about C that other languages like Rust simply cannot do?
Yes. C is the language of the Linux kernel. libc is a fundamental library for interacting with the kernel, and reimplementing it in a non-C language would be a bit of a fool's errand (you'd still be interfacing with C regardless). Of course, you don't have to use libc to use Rust for ex. embedded targets, but if you want your code to compile and run and do useful things on Linux, you're going to have to link against it. And to link against it (or anything on Linux), you'll need a C compiler, and have to talk with the C ABI (see Alexis King's answer for much more in depth about this).
Or is it that existing C code is so well written that there's no point reinventing those wheels in newer languages?
Sometimes. curl is an excellent and battle-tested library despite being written in C. I've seen a number of languages - off the top of my head, some libraries in Nim - prefer to link to curl rather than implement the layers of the networking stack themselves. Rust libraries may do the same, though I do not know for sure.
Does it have anything to do with modern programming languages abstracting away certain primitive programming concepts like pointers/memory, thereby leaving some things "impossible" in the newer language?
No. There are certainly things that are impossible to do in some programming languages: an obvious example is nontermination in languages that are total, like Agda (excluding bugs). If you have certain restrictions on how you want to accomplish a task - for example, if you want to return a reference in a language that disallows it like Lobster, or if you want to have a binary under X bytes in a language known for bigger binaries like Rust, then that certainly can be "impossible", but quite a lot of programming languages that abstract over raw concepts like pointers and memory have some sort of escape hatch to allow you to do what you could do in an unsafe language.
I've heard, anecdotally, that some Python packages rely on C code.
A lot of Python packages rely on C code for performance: for example, numpy. This is because Python is Really Goddamn Slow and less so due to limitations of Python.
So perhaps my question is a more generic "Why does language X have any dependency on C code? Why isn't it language X 'all the way down'?"
There are likely more answers to that that I'm unaware of, but you laid out the questions for the answers I know of nicely in your own post.
(I welcome improvements to this answer. I'm not so confident about my knowledge of libc, in particular.)
open, which fundamentally interact with the system, need to be accessed that way (or by platform- and even version-specific non-standard interfaces, like numbered syscalls; Go does try to do that). The need to move in lockstep means there are very few even replacement libcs for any platform.
$\endgroup$