They should not be ignored, although the modern convention is to treat all consecutive whitespace and comments as a single space. (I took you as asking about this, but some people consider filtering out whitespace a separate earlier step.) So, most programmers would expect
foo /**/ bar to be equivalent to each other, and not to
foobar. In some languages, whitespace has no function but to separate tokens and can always be filtered out of the token stream, but there are languages where that is not true.
Haskell, for example, inserts delimiters into the token stream to open and close blocks determined by whitespace. Another interesting example is Fortress, in which it is a syntax error for whitespace to imply an incorrect order of operations, such as
a+b * c. Modern compilers typically at least warn about code like this, even when the language allows it:
There’s a famous story about a bug found by Fred Webb in a code review at NASA in 1963, with the code for the Mercury space capsule. (An urban legend, which I myself have read in textbooks and repeated in the past, mixes this up with a different bug that caused the Mariner I space probe to explode.) Webb, posting in the Usenet newsgroup
comp.risks in 1990, recalled seeing the line of code
DO 10 I = 1.10
which was an obvious typo for the loop instruction
DO 10 I = 1, 10
In that version of Fortran, there were no reserved keywords, whitespace was ignored (as programmers often did not waste lines on their punch cards with it) and variables could be created implicitly by assigning to them, so the compiler correctly interpreted this line as
DO10I = 1.10
which implicitly declared a
REAL variable named
DO10I and initialized it to
1.1. Although this didn’t cause any catastropic failures in a space flight, it became a well-known cautionary tale, repeated in many textbooks. As a result, mainstream languages since then, including ANSI Fortran, reserve keywords and parse whitespace.
For a long time, it was also fashionable for languages to force programmers to declare all variables at the beginning of their scope (either in a special section like in COBOL or the start of their local block in the Algol family, including classic C). However, modern languages want it to be possible to write static single assignments, so the current trend is to allow variables to be declared almost anywhere, but using a keyword (like
auto in C++17, or
let in Rust) to prevent accidentally declaring a variable with a typo. Today’s compilers will also warn you when a variable is declared but not subsequently used.