Treating strings as character arrays entails a belief that a character array is an adequate model for a string - not just good enough now, but good enough that you won't want to be able to swap it out later while keeping existing interfaces.
In reality, there are many operations on strings that don't make a lot of sense on character arrays, and vice-versa. It's questionable, for example, whether strings should provide random access at all. If you index into a string, should this produce a "character" that is an instance of a separate type? What should one be able to do with that object? Different languages have different answers to this question.
Another problem is that characters are completely different from graphemes. Consider this Python example:
The first string uses a precomposed character to represent the grapheme é with a single code point:
>>> import unicodedata
'LATIN SMALL LETTER E WITH ACUTE'
The second "composes" the symbol using two separate characters:
'COMBINING ACUTE ACCENT'
'LATIN SMALL LETTER E'
There are much stranger phenomena than that:
>>> calendars = "🗓️" + "🗓"
Depending on your environment, the two calendars might look the same or different - for me, they are different in my browser, but the same in my terminal. The length is
3 because the first one is composed of two characters: the calendar emoji followed by a "variation selector". There isn't a "precomposed version" of that; the variation selector is something like a control character that tells the text engine to draw a different glyph (image) for the previous character, while still treating it as the same character.
It is left as a (hopefully simple) exercise to explain why
calendars[::-1] will produce a result that looks the same as
calendars, regardless of the environment.
Don't even get me started on the bidirectional text layout algorithm and associated control codes. You understood correctly: the people responsible for standardizing Unicode are currently on revision 46 of a document about 18 thousand words long just to explain how to decide when to write text left to right and when to write it right to left.
But, sure, reversing a string is a toy problem that also turns out not to be very important in realistic applications. What about the real meat and potatoes of string manipulation?
Did you know that some languages have a concept of uppercase and lowercase, but don't have separate uppercase and lowercase versions of every letter? For example, the German letter ß is considered lowercase; when a word containing it is set in uppercase, the ß is supposed to be replaced with SS (two uppercase Ses)? So it's not actually possible to uppercase words in a locale-aware way by simply iterating over characters and replacing them separately. The length of the string might need to change. I'm not even sure that each character can be considered independently, for every language.
Did you know that many widely-used written languages are only ever intended to appear in a "cursive" form (i.e. there are strokes that visually continue between letters), so there are corresponding variant characters with different appearances (assigned separate code points - not distinguished with variation selectors - perhaps because they had to solve the problem before anyone came up with the idea of variation selectors)? Did you know that you might be expected to substitute the glyphs from those variant characters to render a string, while still treating the string as being composed of whatever was actually put into it? And, if you're the poor sap tasked with implementing text rendering for an operating system, you might be expected to ensure that the characters have their variant appearances when selected, but copying and pasting them gives the original "general" characters?
>>> # Try highlighting, copying and pasting individual letters of the output.
>>> # (You might find it tricky because of the LTR rendering.)
Did you know that the order in which people expect strings to be sorted, is not necessarily based on Unicode code point order at all? And that it might be different for different languages (that could validly have the same two strings in them)? And that it might not be possible to apply naive lexicographical ordering at all (i.e. considering the first character first, breaking ties with the second character etc.)? And that this will get completely broken by combining characters anyway? And that you can't actually necessarily determine the language of text just by examining some characters (
chat can validly be an English verb or a French noun)? And that there's a whole other essay on Unicode.org about how to do it properly? (About 30 thousand words, by the way.)
Did you know that for the East Asian languages that use ideographs and/or logographs (Chinese, Japanese and occasionally Korean - and even Vietnamese, which I didn't know before writing this answer), Unicode provides "unified" characters - such that a symbol might be considered "the same character" in different languages, but is supposed to be displayed differently depending on what language is being written? And that you can't rely on variation selectors to tell you which glyph to use? (And, again, for short excerpts, you can't reliably determine the language by examining the characters - after all, it often takes only a single character to express an entire word/concept?) And that the process of doing this was an absolute ton of work, and the results are still contentious in some places? And that sometimes "traditional Chinese" characters and their "simplified Chinese" counterparts use the same code point (as in the table at the previous link), and sometimes they're separate?
- And that the entire topic can be really politically sensitive?
Did you know that the text in a string could be in more than one language? That it could be in no languages? That it could try to express an idea, despite not consisting of words in a language? That it could try to express an idea that isn't linguistic? That it could try to decorate meaningful text with meaningless symbols?
Did you know about the hack used to cram the Unicode code space into 16-bit encodings? (Or, relatedly, the reason why the allowed number of distinct code points isn't a power of 2?)
Text is really, really complicated. Attempting to model it as merely an array of characters is hopefully naive. While, yes, it is conceptually composed of a sequence of characters (in the only models of it we have that are viable to work with on computers), it makes next to no sense to expose a plain array interface to that sequence (whether or not mutable). Arrays have the conceptual affordance of algorithms that just don't work on text.
This stands completely independent of the question of whether a byte is an adequate model for a character. Yes, it's true that UTF-8 uses a varying number of bytes to encode each character, such that a string implementation relying on a storage buffer in UTF-8 encoding could not be random access. But this is an absolutely trifling issue compared to ones created by naive ideas about what a "character" actually is, and about how text is written, and about what manipulations make sense for that data.
Sure, we have a standard that precisely, unambiguously and permanently maps abstract numbers to "characters". We have a virtual "code space" for those numbers, a commitment not to ever reassign them even if it would make things look much more elegant, and a reasonable assurance that we won't overflow that space for a very, very long time, even as we strain to come up with bizarre pseudo-textual things to fill that space with.
And, sure, there are many plausible ways to represent "a character", depending on system requirements:
A byte whose values correspond to the first 256 Unicode code points (obviously restricted to text that only uses those characters)
A byte whose values are mapped to Unicode code points in an instance-specific manner, according to encoding metadata stored along with the array (restricted to text that only uses characters from one of such single-byte encodings - which can only exist for languages that fit in such a character limit, and don't deal well with mixed-language text)
An unsigned 16-bit value representing a Unicode code point from the Basic Multilingual Plane (only represents those characters - so no emoji, very limited ideographic support, many world languages missing, no private use area)
The same, but using surrogate pairs to represent characters outside the BMP
An 32-bit value, which might as well be signed I suppose
A 21-bit value, bit-packed three to each 64-bit machine word (now that 64-bit architectures are the norm)
And, sure, there might not be any need to use a linked list, or other data structure, to store the underlying character data. Simply writing each character's corresponding number sequentially in memory - i.e., using an array - is perfectly adequate for the simple task of remembering which characters are in the string, in what order.
But... none of these things actually address the fundamental complexity of text. While it's easy to laugh at the previous generation of Western programmers and their naive faith placed in "code pages", the "array" part of "array of characters" is also problematic - perhaps much more so.
Exposing such an interface to the raw storage raises many questions. Simply equating the string type with an array-of-characters type - leaving all that string functionality outside the type itself - causes innumerable problems. Please don't do it in your language.