Why do Rust packages have any dependency on C code?
Taken literally, this question is not really on-topic: it’s a question about some decisions some people made when they wrote some libraries, not a question about the Rust programming language. By and large, their motivations likely have more to do with the practicalities of software engineering than anything fundamental about programming languages. Ultimately, it’s just plain convenient to reuse C code because there’s a lot of it, it’s particularly easy to call into, C toolchains are ubiquitous, and whatever runtime support it needs is almost certainly already loaded into your process.
Still, if we follow that reasoning just a little bit further, there are definitely a few programming language questions hidden in there. For example, why is C so exceptionally easy to call from other languages? Why does Rust have built-in support for C calling conventions and struct layout but not other languages? This is by no means specific to Rust: in most languages, “foreign function interface” might as well be a synonym for “a way to call into C”, and it is not at all uncommon for C toolchains to show up in unexpected places.
Why is that?
The phrase “the C ABI” is something of a misnomer, as C does not specify an ABI, nor is it really defined at the level that would allow specifying one. Rather, “the C ABI” refers to a general collection of things defined for every combination of architecture and operating system. For example:
The calling convention specifies a standard set of rules for how arguments are passed to and returned from functions, the layout of the stack, and what registers are preserved across the call.
Datatype size and alignment rules specify the necessary size and alignment of primitive types like
Structure type layout rules specify how composite structures are laid out in memory and how they are passed to and returned from functions.
Executable formats define how executables and shared libraries should be laid out so that the operating system knows how to load them into memory, set up their virtual address space, and dynamically link them.
It is easy to quibble about what is and is not part of “the C ABI” for a given platform. Are executable formats really a part of the C ABI, or are they something else? What about thread local variables? Signal handling? Debugging symbols and unwinding information? I’m not going to try to pin down some precise list because there simply isn’t one. What matters is two things:
Where precisely the C ABI and C standard library end and your operating system begins is not at all clear.
It is extraordinarily important that the C ABI for a given platform does not ever change.
These two things are very closely related.
C is your operating system’s interface
As apropos’ answer points out, there is a real sense in which C isn’t (just) a programming language. A staggering amount of essential operating system functionality is not just most convenient to access via C libraries, it’s the only supported way of accessing it. Sure, if you really want, you can find the conventions for making a syscall “directly”, but the idea that syscalls are the interface between userland and the operating system is rather idealistic. For one, syscalls are the interface between userland and the kernel, and the kernel is really an exceedingly small part of even a fairly minimal modern operating system. For another, they are poorly documented and require reverse-engineering on proprietary operating systems, and worse, syscalls are not even stable on Windows.
So it is really rather black and white: the intended way to call into your operating system is to call a C function. In fact, let me make this even more explicit: it is not just the intended way for a human to write a computer program that calls into your operating system, it is the intended mechanism by which every programming language in existence calls into your operating system. On Linux, one can perhaps quibble with this (though I still think it would be quite ill-advised), but on macOS and Windows, you really cannot, so there’s just no way around it—your programming language needs some way to (eventually) make C calls.
That said, if your programming language is interpreted, you basically don’t have to worry about this problem. Whatever language your interpreter is written in already knows how to make the necessary calls, so you can just defer to that. In fact, even a compiled language that uses its own binary object format and does its own linking and loading and has its own runtime can largely get away with not knowing how to generate C calls because the program can always yield to the runtime’s scheduler, which is written in a language that knows how to perform the call into C.
This means that, counterintuitively, being a systems language without a runtime (like Rust) means you actually have to care more about the details of the C ABI on each and every platform because you have to be able to generate code that makes those calls directly. You have to speak the platform’s calling convention, you have to do all your stack layout in a way that’s compatible with the C stack, you need mechanisms for generating code that uses the correct structure layout and alignment rules, and you need to figure out what to do when a function written in your language returns to a function written in C, something that is not altogether obvious when your language provides functionality like exceptions that is not supported by the C ABI.
C ABIs are difficult to separate from the C language
After everything I said above, you hopefully understand why just about every programming language under the sun needs some way to speak the platform’s C ABI, but you might wonder whether calling it “the C ABI” is very useful. You might think it would be clearer if we called it “the platform ABI,” and in fact that is often done! This might lead you to believe that we could dispense with all the programming language parts of C—like type declarations and header files and
.o objects—and just focus on the binary interface bits, which really don’t need to be considered through the lens of C at all.
Sadly, if you actually try to implement a compiler, you will rapidly discover that this ABI is painfully difficult to disentangle from the programming language C. In order to make a C call, you don’t just need to know what registers to pass arguments in and what size an
int is, you need to know things like struct layout and what it means to spill a struct onto the stack. You also need some way to take the interface specifications provided by operating systems and turn them into a format your compiler can work with, and wouldn’t you know it, those interface specifications are provided in the form of… C header files.
No, really: go down this rabbit hole and it will start to drive you mad. Pick just about any syscall that involves a struct type and try to figure out how to automatically determine how to generate the right code to use it. Consider the case of
stat. The man page says it’s in
sys/stat.h, but how do we even find where that header is on some arbitrary system? The highest-voted answer to a Stack Overflow question about how to do this on Linux is to call
gcc. I assure you it does not get any easier to avoid giving in and calling a C compiler from here.
The reality is that C header files are currently used as the lowest-level interface description language you can get your hands on, and this is a somewhat frustrating choice seeing as it is virtually impossible to parse them without writing a C compiler. But let’s suppose you did that—you deduced all the platform-specific rules for layout and calling conventions, you reimplemented the C preprocessor, and you wrote a fully-featured C parser and symbol resolver that allows you to parse any standard C header file—you will rapidly run into another problem.
The header files on your system are not written in standard C.
C ABIs are difficult to separate from C toolchains
This part can come as a bit of a shock to people, especially since there’s some idea that the header files found on your computer are compiler-agnostic. In fact, if you go digging through your system’s header files, you’ll find an awful lot of
#ifdef __GNU_C__ and
#ifdef __clang_major__ littered about. In practice, your header files are very much vendor-specific, they just explicitly handle a small list of known vendors.
At this point it may become clear how hopeless this all is. The deeper you look, the fuzzier the line between your operating system and the C compiler becomes. In a way, the entire C toolchain is a part of the operating system, the wildly complex decoder ring you are forced to use if you want to be able to conform to the eye-wateringly elaborate machinations required to call into your operating system. Yes, I realize this might sound hysterical, but consider just how deeply the C toolchain is interwoven with languages that, frankly, would rather have as little to do with it as possible:
There will always be people who quixotically struggle to avoid depending on the C toolchain at all costs, but doing so is essentially hopeless. Even if one were to implement everything required to consume system header files without MSVC, GCC, or Clang, support for those headers is such a moving target that it would likely create a headache-inducing maintenance burden. What’s more, C’s exceptional ubiquity only entrenches it further: when software written in different languages needs to coexist in the same process, the mediator is the collection of languages, tools, runtime infrastructure, and operating system support that we now know under the moniker “C”. Why not something else? Because everything already supports C. So everything has to support C.
To quote the current Twitter bio of JeanHeyd Meneide, current Project Editor for the C standardization committee:
The C Standard Cannot Be Replaced And Will Never Be Destroyed.