26
$\begingroup$

The discussion whether using indentation for code blocks is better or worse than using braces is an old one, but I don't want to rehash that discussion here (for reference, there was a fairly recent question about the drawbacks of indentation that summarizes some of the arguments).

Instead, I'd like to know if there has ever been a systematic study on which of the two design choices is easier to learn for absolute programming beginners. I'm thinking of something like a study in which one group of participants (without any prior experience in programming) was taught a proto-language that featured code-block indentation. The other group of participants was taught a syntactically equivalent proto-language with the only difference that this language and the teaching material used braces instead of indentation. You could then compare the success of these participants in performing little coding tasks.

I've searched for such a study, but was unsuccessful. Has any such study ever been published?

$\endgroup$
9
  • 3
    $\begingroup$ I will come back to this tomorrow, but I believe the answer is no, there hasn't been any systematic pedagogical study of that specific feature in isolation (in all honesty, it's likely infeasible to do so in any reliable way for one individual element). There is evidence that use of indentation in addition to other structure is helpful, although a little more mixed than one might expect; there are also comparative studies on structured editors where "indentation" is innate compared with non-structured editors for the same language, but those have significant confounds. $\endgroup$
    – Michael Homer
    Nov 28, 2023 at 9:02
  • 3
    $\begingroup$ Just to clarify, when you say "braces", do you mean "braces with their content indented" (as in C-like languages), or "braces but no indentation"? $\endgroup$
    – Pablo H
    Nov 28, 2023 at 16:55
  • 2
    $\begingroup$ @PabloH I believe it is safe to assume that the OP meant braces with indented content, otherwise it wouldn't have been too much of a competition versus plain indentation. $\endgroup$
    – Hg0428
    Nov 28, 2023 at 17:38
  • 3
    $\begingroup$ @Hg0428 Exactly. But then the question is "having always indentation, braces or no braces?". $\endgroup$
    – Pablo H
    Nov 28, 2023 at 17:49
  • 3
    $\begingroup$ I don't think the study you propose would necessarily be fair, because even in languages with braces, beginners are (appropriately!) taught to also use indentation to denote blocks. And it seems obvious enough that braces without indentation would be terrible. So really it's about whether the small difficulty of writing braces is worth the small benefit of your code working if you get the braces right but the indentation wrong. I'd argue the latter is not a benefit, because if a beginner has indentation not matching their braces, the program likely doesn't do what they want it to ayway. $\endgroup$
    – kaya3
    Nov 28, 2023 at 18:08

2 Answers 2

24
$\begingroup$

I don't believe that there is any systematic pedagogical study on this feature specifically. Studies of actual learning are complex, time-consuming, and expensive, and I don't think any has been done (or likely would be) on that difference in isolation. The totality of human-factors studies comparing language design decisions likely numbers in the dozens (depending on your counting criteria, somewhere between 35 and 137 existed in 2012, per Antti-Juhani Kaijanaho's PhD thesis).

However, there has been work on program comprehension and reproduction under different indentation and delimiting modalities, and on learning performance in different languages and editing environments. These are more mixed on indentation than you might expect: it's widely accepted that correctly indenting to match structure is helpful, but in fact empirical studies often show limited or no impact in practice. My high-level summary of the state of the art for block structure for novice programmers is that

  1. indentation is valuable, but they don't use it unless enforced; and
  2. keyword-based systems are more effective than symbol-based systems;
  3. redundancy enables enhanced errors and thus improves effectiveness.

None of these are strongly results. It is not clear that the benefits of keywords survive to expert programmers or with editor support for delimiting symbols. There has also been a lot of work with Lisps for novices, which are all delimiter and no indentation. It is likely that the ideal system for learners at this stage will both use delimiters and enforce that indentation is consistent with those delimiters (and this is the approach that recent novice-targeting languages like Pyret and Grace have taken, albeit without specific evidentiary basis).

I will point to some specific studies below, although they are not as strongly-grounded as I would like. None of the indentation-focused ones are comparing to brace-based languages or to semantically-significant indentation, though there are some suggestive pointers. I think the most relevant result is from the Quorum studies, suggesting that braces in particular perform poorly.


These studies looked at Pascal (keyword blocks, but all begin-end) programs with different indentation levels. The findings were that moderate indentation was beneficial for novice comprehension over both no indentation and greater levels of indentation. These are not brace-based programs and participants were already learning or programming in Pascal; the effect size is also surprisingly large for the difference between four- and six-space indents. There were a number of indentation studies around this time, with generally similar results, but not with novices in pure indentation-structured languages.

This was a partial replication of the previous study, using curly-brace Java instead of Pascal, with students, and eight instead of six spaces. They found no impact from indentation at all on any of their measures: even zero indentation gave similar performance.

This compares an IF-GOTO-based structure with (non-semantic) indentation with a variety of block-structured systems, using genuine novices in a controlled trial. The closest to braces is the begin-end variant of the language, and the programs in the GOTO version looked like an off-side-rule language, but the line labels were semantically significant; the other variants used IF-OTHERWISE and IF x-NOT x. The methodology had participants review a short tutorial on the language condition in use, and then write programs to implement a specific condition table. Indentation helped for all but one of their variants, but the version with distinguished start and end markers for each conditional performed worse. Participants had a lot of difficulty getting the indentation right in the GOTO case, but when they did their programs were no worse than others.

That's about it as far as studies that isolated indentation as a factor, and none of them are really approaching your question directly. We can take some hints that indentation is probably useful and can be sufficient, but that delimited block structure can also be sufficient on its own.

This is part of the Quorum project, a focal point for modern empirical language design for novices (and the source of the famous "randomly-generated language more usable than Perl" claim). Most of this relates to keyword choices and indentation was not studied specifically, but of relevance here was a finding that block constructions that did not use braces had better performance metrics than those that did, although that subgroup included both the indentation-only Python and the keyword-end Quorum and Ruby.

The best performers, very close together, were Ruby and then Python, with Perl and Java not statistically significantly different from Randomo, which was Quorum with braces and random ASCII keywords. Some issues writing the end keyword correctly were reported, but Quorum still uses that keyword today, so it wasn't found to be that bad of an issue.

In all cases, "correct" indentation was presented in the learning material, but I don't believe it was required in what was produced except for Python. Indentation is encouraged but not required in Quorum, I think to reduce unnecessary errors during editing. Follow-up work on Quorum has reinforced these results for it.

There are various other studies on teaching language pairs that I'm sure you've seen, but they don't generally get to this level of granularity and there are significant confounds for teasing out anything like this.


To conclude here, I don't believe such a study exists. I am confident that there isn't one prior to 2014, and I am not aware of and couldn't find any since either. I am sceptical that it would be worthwhile to run one for novices, because the value of that one factor is unlikely to be worth the substantial cost of running a genuine learning study, though a comprehension study is plausible.

The general direction of the literature is that braces are not highly effective for novices, although lexically substituting an explicit end-of-block delimiter word for } seems to do better, and omitting any opening delimiter has some support too. Semantic indentation isn't supported one way or the other by this, but there is some evidence that matching end delimiters presents a difficulty, which is a matter that doesn't arise for semantic indentation. I don't think there is sufficient evidence to draw a conclusion one way or another.

$\endgroup$
5
  • 2
    $\begingroup$ How did you find all this???? Google and Bing could find nothing on the subject. Great answer by the way, well formatted and very useful. $\endgroup$
    – Hg0428
    Nov 29, 2023 at 17:03
  • 1
    $\begingroup$ @Hg0428 I looked in the ACM DL in the first instance, but I’d seen most of them before so I knew they existed. Some backward/forwards snowballing from one I’d already found, but there really isn’t very much to find, I think. $\endgroup$
    – Michael Homer
    Nov 29, 2023 at 17:11
  • $\begingroup$ This is an awesome answer – much more thorough than anything I'd imagined. Thanks a lot! $\endgroup$
    – Schmuddi
    Nov 29, 2023 at 21:12
  • $\begingroup$ In the lisp/scheme cultures that I’m aware of, poor indentation is a faux-pas. That is, it is encouraged to aid comprehension but not required to compile (which seems similar to what you say about Quorum). $\endgroup$ Nov 30, 2023 at 14:09
  • $\begingroup$ Fortran-style end-of-block delimiting seems to be an especially worthwhile point in the design space, given your numbered summary points. For anyone unfamiliar, the only required syntactic element is end, but it can optionally be elaborated in agreement with the thing being ended. if/end if, subroutine foo/end subroutine/end subroutine foo. That might be nice for Quorum to try if they haven't already $\endgroup$ Dec 2, 2023 at 3:06
2
$\begingroup$

As far as I know, no such study exists. I haven't been able to find one. But, to attempt to help you as much as possible, I will tell you what I have learned on the subject.

In my experience teaching and helping beginners, indentation provides a more readable and intuitive interface for the beginners, yet many will begin to prefer braces once they get used to coding. Both have an initial learning curve, but it is initially a little higher with the braces, because the new coder has to get used to the idea of using braces to group things. Indentation is a little easier in this respect, because everyone is already used to grouping things with indentation. Indentation is everywhere, from paragraphs to quotation. It's not as much to get used to, initially.

Overall, I don't think there is one choice that is definitively better for beginners. To make a study on the subject would probably be impractical because there would likely be more difference in how well the two groups learned than the difference actually made by the type of code block.

$\endgroup$
2
  • $\begingroup$ Could I add to this by saying that GNU brace placement is generally far easier for students to get used to than K&R $\endgroup$ Nov 28, 2023 at 17:32
  • $\begingroup$ ^^ emacs style, not GNU style $\endgroup$ Nov 28, 2023 at 17:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .