Why would accessing uninitialized memory necessarily be undefined behavior?

Question

In C, accessing any indeterminate/uninitialized memory is undefined behavior, period. Even in the case that the type in question is guaranteed to have no trap representations, such as unsigned char or any of the optional fixed-width integer types.

On of the reasons I thought of as to why uninitialized memory access is undefined, is because there might be a trap representation. But if the type being used to access the memory has none, this is irrelevant.

For what purpose would a language designer declare any uninitialized memory access undefined behavior (program behavior is meaningless) as opposed to merely unspecified (the value contained could be one of any number of possible values unpredictably), especially with types that do not have trap representations?

Answer 1

It's not strictly necessary for uninitialised memory access to be undefined behaviour in the "nasal demons" sense. A language which allows raw memory access could still choose to specify some behaviour for otherwise-valid (i.e. properly aligned, no page faults, etc.) reads from memory ─ for example, it could result in an unspecified value for just that expression, without rendering the whole program as UB. Indeed, machine code itself is such a language.

The reason why C makes it UB is more to do with C's design philosophy than necessity. C is designed on the basis that the programmer (1) knows what they are doing, and (2) always wants the compiled output to be as efficient as possible.

So the language design logic is something like this: reading from uninitialised memory is so rarely useful that the language need not allow the programmer to do it. Then by (1) it's assumed that the programmer will take care that their code will never do it. And therefore, even if it looks to the compiler like a program could do that, the compiler is permitted to assume that it doesn't, which is good for (2) because the extra assumption might permit better object code to be generated.

---

Now, we all know that this basically amounts to the unrealistic assumption that the programmer knows the C language spec inside-out and never makes mistakes; and real programs have had critical bugs (including serious security vulnerabilities) due to code written under the misapprehension that it's OK to read uninitialised memory if you don't use the value for anything. So to a large extent this kind of thing is a design flaw in the C programming language; and most modern languages are designed to be memory-safe on the basis that real programmers aren't perfect and do make mistakes.

The counterpoint is that languages designed like this are needed for the same reason professional chefs need very sharp knives ─ they allow experts to do things that aren't possible with safer tools. The fact that the sharpest tools aren't suitable for all users or all uses doesn't necessarily mean that those tools are badly designed. I respect this argument, but it's ahistorical; C was designed to be a general-purpose language for widespread use.

In fact there are lots of things in the C standard that are or have been UB which don't need to be. One of the most egregious examples is

An unmatched ' or " character is encountered on a logical source line during tokenization

If anyone knows of a sane reason that an unclosed string or character literal should be UB rather than a syntax error, I would love to hear it.

---

As a side-note, since the question is about reading uninitialised raw memory, it's worth adding that a raw write to arbitrary memory really does have to be UB (if the language allows it at all); the memory written to might be the call stack, the program counter, or the executable code itself, so such a program really could have arbitrary behaviour (depending on the machine running the compiled code).

Answer 2

On top of what everyone else has said, UB is also often used to signal the fact that a program could mean different things at different optimisation levels.

Consider this example:

int
foo(int x)
{
    int y;
    if (x > 42) {
        y = 5;
    }
    return y;
}

If you call foo(3), it might return whatever value was already in the location (whether register or stack) where y happens to be assigned. The returned value could be different on different calls. This function, even though it doesn't mutate or read "global state", is not referentially transparent.

But an optimising compiler might optimise it to this:

int
foo(int x)
{
    return 5;
}

This is a completely valid transformation. By using an uninitialised variable, you, the programmer, effectively told the compiler that you did not care what value was returned if x <= 42. If you don't care, then the compiler gets to pick any value it wants.

By the way, if you're curious how to implement this in a compiler, it's quite straightforward if you already know about abstract interpretation. Typically, we use the bottom element $\bot$ of the domain lattice as "don't know", but what does the top element $\top$ mean?

It turns out it means, more or less, "don't care", and this is exactly the situation where that is useful. After the if-then join point, you take the greatest lower bound of $\top$ and $\{ 5\}$, and get $\{5\}$. The compiler can then conclude that assuming that y always equals $5$ is safe.

It's also legal for a compiler to assign a "Heisenberg" value to an uninitialised variable: it doesn't have a fixed value until you look. Such "values" don't behave like a variable which was initialised with one (possibly random) value when it was created.

Here's a highly artificial example:

int
foo(int x)
{
    int y;
    if (x < 100) {
        if (x < 50) {
            y = 5;
        }
    }
    else {
        if (x < 150) {
            y = 10;
        }
    }
    return y;
}

This is a legal optimisation:

int
foo(int x)
{
    int y;
    if (x < 100) {
        y = 5;
    }
    else {
        y = 10;
    }
    return y;
}

But there is no value that you could "initially" assign to y that would result in this optimised code.

Answer 3

Many instances of undefined behaviour in the C specification represent accommodations for existing implementations where the "obvious" treatment was not possible, including detecting and diagnosing it. For example, reading a pointer to uninitialised memory may trigger memory-mapped effects on some architectures, beyond merely producing an unspecified result, or local variables may run into hardware guards on register access. These are exactly the "nasal demons" scenarios where effects outside the language can occur. These platforms are less common today, though many embedded architectures do have unusual behaviours to someone accustomed to x86 or ARM.

A language that wishes to compile directly to these platforms, or a specification for a language with existing implementations on them, will need to address these situations. The C specification is in this category, and encoded the commonality of what existed, not a from-scratch definition of what ought to be. It gives some minimum portable guarantees, but also accommodated the (possibly bad) choices made already.

A new language probably doesn't need to have any of this. These sorts of systems may be irrelevant enough to rule out, or it may be able to define its memory model sufficiently to avoid this coming up, requiring implementations for those targets to do extra work to accommodate it. It may just make uninitialised access inexpressible at the language level. Given the much-reduced compatibility costs, it would probably be the right choice.

Answer 4

I am not sure if this was in the C standardisation committee's mind when making this decision, but it is worth noting that some computers may perform arbitrary actions when memory is read. A memory read is generally translated by the processor into a signal on a bus that is connected to that processor, and devices connected to the bus may behave however they wish in conjunction with that. Thus reading from an arbitrary address may have arbitrary effects.

One real-world example is the Apple II computer, which had various addresses that changed system behaviour when read from, which were typically called "soft switches". Reading from addresses in the range 0xC010-0xC017 will cause various sections of memory to be paged in and out (potentially including the memory containing the code being executed). Reading from addresses between 0xC018-0xC01F will change display modes, etc.

A more modern example might be memory-mapped registers in various microcontrollers, for example in AVR microcontrollers such as the ATtiny402 there are a number of addresses that correspond to the low byte of a 16-bit register. When these are read, a different memory address's value (the TEMP register) changes to the current value of the high byte of the same register, thus allowing two different 8-bit read operations to reliably access a 16-bit value that may change asynchronously.

I'm not sure what the best way the C specification could have used to specify that this kind of behaviour could happen, but it is important to note that memory reads are not necessarily entirely without side effects.

Answer 5

For what purpose would a language designer declare any uninitialized memory access undefined behavior (program behavior is meaningless) as opposed to merely unspecified (the value contained could be one of any number of possible values unpredictably)

A big question is how you obtain a pointer to the uninitialized memory.

The simplest case is an uninitialized stack variable. Modern systems allocate stack page-by-page on demand. Every system I know does so on any access, but it is conceivable that some system could do it only on first write. That could even be useful to catch some out-of-bounds read errors, and throw a segmentation fault.

The way C standard is written, it avoids over-defining behaviors when there is no useful purpose for them. It's hardly more useful to read an unspecified value than to potentially crash, thus it is not defined.

Note that this does not limit implementations: any implementation can define that access to uninitialized stack variable successfully reads unspecified value. If the implementation does stack scrubbing, it could even specify that uninitialized stack variables read out as zeros. Sadly, many implementations do not have good documentation on such behaviors.

Answer 6

TL;DR: It's golden for Static Analysis.

A dreadful example

One of the issues with defined behavior, is that readers can no longer distinguish between accidents and intent.

For example, in C, unsigned integers are defined to wrap on overflow. This is useful in a number of cases: when computing a hash (just add) or when "temporarily" overflowing (a - b + c = a + c - b). But there's also plenty of cases where overflows are just errors, some of them leading to catastrophic failures (malloc(n_items * sizeof *items) being particular bad => use calloc instead).

The problem, though, is that because it has been defined, a reader cannot know, for certain, whether the overflow is intended (and fine) or accidental. And any static analysis tool (compiler, linter, ...) warning about every overflow will get thrown out of the window because it will have way too many false positives.

Reading from uninitialized memory

In general, reading from uninitialized memory is most likely to be a bug, an accident, and therefore it is good that it be spotted by readers, and detected by tools, as the accident that it is.

Making it UB is one such way to ensure that no reader, nor tool, will ever have the slightest doubt that it could potentially be intended, instead of the accident that it is.

Alternative: Debug vs Release

The Rust language offers a more interesting alternative.

In its quest to eliminate UB, the Rust language was confronted to the integer overflow issue. Unfortunately, trapping on overflow results in significant penalties even in optimized binaries, not only because LLVM only includes the detection for debugging purposes, but also because trapping on overflow breaks fundamental properties -- such as associativity of addition or multiplication -- which in turn prevent many optimizations. In fact, even auto-vectorization is impossible in many cases, because vector instructions only offer wrap-on-overflow behavior :(

Faced with UB on overflow vs Wrap on overflow... the Rust language toyed with the idea of yielding an unspecified value on overflow, and finally settled on:

• Wrap on overflow, in Release, by default.
• Panic on overflow, in Debug, by default.

(Plus some verbiage that panicking may become the default behavior in Release at a later point, as compilers evolve, and a flag to override the defaults)

The exact options selected matter less, for the purpose of generalizing the rule, than the divergence between Debug and Release behavior. This divergence of behavior essentially codifies that despite being specified, overflow is necessarily an accident.

How to apply this divergent behavior to reads from uninitialized memory is left as an exercise to the reader.

Answer 7

If nothing else, it simplifies the language specification, and as a consequence makes it easier for programmers to understand the language, and also makes things easier for implementors.

As you say, some types may have no trap representations, but others might. But distinguishing access to uninitialized variables based on the type adds complexity to the specification. Programmers will have to take note of the types of their variables when taking advantage of being allowed to access them. Compiler writers will need different optimization code depending on the variable types.

And what do we gain from this added complexity? Not much. There's rarely an important reason to access uninitialized memory. An exception might be something like assigning a struct that has some uninitialized members, or memcpy() a buffer that may only have a portion filled in (although the application should usually know where the initialized portion ends and can use that as the copy size).

In practice, nothing bad is actually likely to happen from these accesses. While it may be UB to use memcpy() when some portion of the copied memory is uninitialized, it's not UB to use realloc() on a pointer to uninitialized data. Yet realloc() effectively has to do the same memcpy() when it needs to relocate the allocation -- there are few architectures where realloc() can tell which portions are allocated.

Answer 8

The "as if" rule means that a variable need not have a fixed address in memory unless you take its address (and keep that in a pointer variable somewhere else).

This can be used by optimizers to store the "current" value of a single variable in a succession of different locations, including multiple registers, and maybe even multiple memory addresses (*1).

So when some branch of the code causes the compiler to ask its register/variable tracker "where is the value of this variable", and the data flow analysis proves that it's uninitialised, the tracker isn't allowed to say "dunno" (because that would abort the compilation), so it may simply choose a random register and say "in here".

Even if it always picks the same dummy register, it might also get used for something else, so the apparent value of the variable isn't just "unspecified", it's random every time it's read.

For example, given this:

extern int foo();
void func() {
  int a;
  if (a && foo() && !a)
    puts("a is uninitialised");
}

clearly foo can't see a, nor intentionally alter it, and yet the apparent value of a could change because foo is compiled to something that uses the register that also gets used when a is read. Since a doesn't "own" the register, it needn't be saved while foo is called.

At the same time, the compiler could look at a && ... !a and decide that that can "never happen", and remove the code for the puts statement entirely. So when foo is called and comes back with true, there are random bytes where the program code should be. Then we get to watch the parade of nasal daemons.

(*1: Some CPU's can do PUSH and POP faster than general memory writing & reading, and the compiler could take advantage of that when it needs to spill registers, rather than always writing to locations based on which variable the register currently holds.)

Answer 9

Jens Gustedt, who is involved in the C standardization, has addressed this in an answer on SO.

The executive summary:

Reading uninitialized memory is not necessarily undefined behavior in C.

There are exactly two things that can go wrong.

1. The type has trap representations, and that happens to be the representation in that memory location;
2. An object with automatic storage duration (i.e., a local variable or a parameter) may be held in a register. The reason this is UB is that there are platforms which have special provisions to prevent read before write from registers, as an error detection.

UB is restricted to these two possibilities. We know that there are no trap representations. If your variable does not have automatic storage duration, or if you take its address — which prevents it from being in a register — you are safe.

Answer 10

The only way in which the C Standard can accommodate situations where optimizing transforms may yield program behavior that's observably inconsistent with sequentially executing a program as described by a platform's abstraction model, is to categorize as Undefined Behavior at least one action performed in a program execution where that may occur. This makes it necessary to classify as Undefined Behavior actions whose behavior might be affected by optimization, even if the original and transformed behaviors would have satisfied application requirements.

Consider the following pair of functions:

struct blob { uint32_t ct; uint8_t dat[28]; } x,y;
void test1(void)
{
  struct blob temp;
  temp.ct = 3;
  temp.dat[0] = 10;
  temp.dat[1] = 20;
  temp.dat[2] = 30;
  x = temp;
  y = temp;
}
void test2(void)
{
  test1();
  fwrite(&x, 32, 1, someFile);
  fwrite(&y, 32, 1, someFile);
}

What should or shouldn't be guaranteed about the two blobs written to the file?

If nothing in the universe would care about the contents of the last 25 bytes of each object, the most efficient machine code implementation of test() that would satisfy application requirements would probably write the count field and first four bytes of data into both x and y, while leaving the other parts of those structures unmodified (though it might in some cases be faster to only write the first seven bytes of the structure). The C Standard, however, would have no way of allowing the last 25 bytes of each record from being written differently without allowing implementations to behave in completely arbitrary fashion.