Avoiding student confusion about the meaning of `x == a or b`

Question

I'm designing a language intended to teach kids and other beginners how to program. It's just a fork of Python with a few extra features to make it easier to learn.

I've noticed students make one specific mistake very often: They'll write code like this:

number = input()
if number == 5 or 12:
    print "You guessed the correct number!"

This of course won't work of course, since it parses as (number == 5) or 12 and 12 is always truthy, thus you will always enter the if regardless of the value of number.

Students make this mistake very often, and it's especially deadly because it does not produce any kind of error, and since they are not very experienced in debugging they can spend days unable to track down the issue before they ask for help.

I could of course special case this syntax and transform it into number in (5, 12), but the goal here is not to just make it work but help the students build an intuition for how expressions and statements work, and ease the transition to real python later in the course.

Are there any changes to the language I could make to make this syntax more intuitive or make the error easier to spot while ideally staying somewhat close to python?

Answer 1

Pyret is a Python-like language explicitly designed for education. This is what it produces on your example program:

Note that the error message doesn’t explicitly call out the logical mistake, and perhaps it could. Pyret does have some special-purpose error reporting technology for other sorts of common mistakes. But the simple solution they’ve used here instead is to simply make == and or have incompatible precedence, so programmers have to be explicit about their grouping.

But suppose the user simply adds parentheses around 5 or 12. Pyret then produces this error:

Note that logical operators are restricted to booleans. This catches mistakes earlier, and it helps the programmer understand that the program isn’t behaving the way they think.

These examples illustrate that some work making basic error messages as helpful as possible can go a really long way. The authors of Pyret have implemented a development environment with color-coded highlighting to help illustrate how the errors correspond to portions of the user’s program, and they’ve also presented them in a way that is likely to be less overwhelming to beginners. Many of these decisions were empirically supported by user studies, published in the paper Mind your language: on novices' interactions with error messages (pdf).

As your question states, leaving some of the details up to students to figure out for themselves could help to develop some programming problem-solving skills. The important thing is presenting the information in a way that makes it most likely that the students can put the pieces together.

Answer 2

I suspect the problem here is that the Englishy keyword or seems like it groups values as in number in set(5, 12) but is actually a connector for boolean expressions.

Being strict about both sides of or being a boolean expression can help catch these kinds of problems earlier. I've written more about some pitfalls of boolean coercion elsewhere. Even if your language is untyped, first making sure that you have some kind of type-like error at least at runtime is a place to start, and then you can try to customize the error messages as you learn more about the ways your user population goes astray.

It's nice that you're working on a teaching variant of Python which started off as a fork of ABC, a BASIC variant also meant for teaching.

"The Programmer's Brain" tackles the topic of how we read code, touching on human cognition and natural language use, and has quite a bit on how beginning programmers are led astray.

I believe Felienne Hermans, the author of that book, has a teaching language, Hedy and I think it's also trying to build intuitions that mesh with Python. Maybe seeing how one expresses ideas that your users have trouble with in Hedy would give some ideas.

Answer 3

Expressions containing AND and OR should not contain any part that is always truthy, or falsey - this should produce a warning or even an error. If it's for learning, maybe errors are scary, a tip might be better.

TIP: 5 OR 12 always equals 5

TIP: 5 AND 12 always equals 12

Answer 4

You could issue a warning when an integer literal (this is important -- using an integer variable or a function result in this way can absolutely be legitimate) is used in a boolean context, i.e. in an if condition, or as an argument to and, or or not.

But on a larger scale, you might want to look into partial operator precedence ordering. This is a slightly weird use case for it, but it will still be immensly helpful. Essentially, instead of associating a priority with every operator (which in modern Python is done implicitly by just nesting the grammar rules in a certain way), you say that some operators' precedences are not comparable. For example, this could be done for and and or. Then the construct a and b or c would be invalid, and the programmer would have to explicitly specify (a and b) or c or a and (b or c). For your case, you want comparisons to be 'parallel' to or, and perhaps and. Then the hypothetical student making that mistake would probably write if number == (5 or 12):, which would still result in a silent mistake, but of a different, more easily observable sort: 5 would work, but 12 wouldn't. Hopefully, the reasons behind this would be more apparent to a student.

In my language, I'm actually using the same (albeit modified heavily, but keeping the syntax) parser generator as Python 3.9+ does. I actually employ partial precedence ordering, and you can see examples of it in the grammar definition. Hopefully that should help implement it for yourself

Answer 5

Keep your types separated, only True or False are legitimate boolean values, and make sure that or only works with boolean. This is for example how Scratch does it, with different shapes for different types of values. Since 12 cannot be used as an operand for or, your interpreter will detect it.

Alternatively, you may want to make parentheses mandatory around logical operators to avoid confusion with and vs or too, as in:

if x == 3 and y == 2 or y == 5:
    pass

Answer 6

Use any/all operators and compound operators instead

I could of course special case this syntax and transform it into number in (5, 12), but the goal here is not to just make it work but help the students build an intuition for how expressions and statements work, and ease the transition to real python later in the course.

Even so, this is langdev.SE, so I want to explore this possibility. Since "special cases aren't special enough to break the rules", I'd like to propose something more self-consistent.

This approach is inspired by the built-in functions any and all in Python - what if they were operators instead, and replaced and and or (so as to avoid confusion by trying to use both at once)?

x OP any a, b, c would have semantics equivalent to (x OP a) or (x OP b) or (x OP c). Similarly for all with and. You could perhaps require OP any to be joined up into a single token. Either way, the fact that the single keyword any is not infix, and is in the same place as the operator, avoids the weirdness of seeming special-case semantics. (Of course, it would be possible to do this with conventional or and and, but such multifix operators are harder to implement.)

Similarly, just any a, b, c could mean a or b or c - there's no OP, so nothing to apply separately to each value.

To recover the functionality of Python's builtin any/all functions, then, just allow Python-like unpacking generalizations to work with the expression list:

# python
any(f(x) == 3 for x in ys)
# new language
3 == any *(f(x) for x in ys)
# or even:
3 == any *map(f, ys)

The hard part is making it short-circuit ;)

Answer 7

I think there's a philosophical question to be considered here: is it better for a teaching language to be "Python-like", or for it to parse statements properly so that == and or are treated with the correct precedence?

One way round it would be to avoid the issue by using extremely strict type checking, and to never apply automatic conversions which are an unending source of problems.

I'd add that experience has led me to believe that = should be "considered harmful", use == for comparison and := for assignment with a simple = being an error in all cases.

But again, the question remains of how closely a teaching language should stick to Python syntax and conventions: the simplest solution, which could be applied without syntax changes, would be rigorous type checking.

Edited to add: as an historical note, https://en.wikipedia.org/wiki/Plankalk%C3%BCl#Assignment_operation describes controversy relating to whether Zuse favoured as an assignment operator, which became ← (supported by ASCII-63 and several 6-bit character sets) or the digraph :=

Answer 8

The underlying problem is that, unlike COBOL, python doesn't try to make sense when read as English.

COBOL was designed so that non-technical managers could mostly understand the code, and it worked well: One doesn't need a degree in mathematics to understand the meaning of a IS GREATER THAN b AND c, OR EQUALS d.

On the other hand, python is happy to combine different things in strange ways that make sense mathematically, but not to the "common reasonable man", who is not going to be able to understand that if 12 has any sensible meaning.

Not surprisingly, kids and beginner programmers don't think like mathematicians (who can blindly accept rules like "== binds more tightly than OR").

You might consider requiring that for a or b, a and b must have the same type of semantics.
Or better yet (as suggested by @coredump), don't allow implicit conversion to Boolean.
E.g. reject Python's if x > y or z and require that it be entered as if x > y or z != 0.

Similarly reject expressions that in English either are ambiguous or have different interpretations.
E.g. x+y*a+b should be an error, requiring (x+y)*(a+b) or x+(y*a)+b.

(To be honest, I think those would be good changes to make to the real Python, but I won't hold my breath.)

Answer 9

It may have something to do with my natural language isn't similar to English, but when I learned, I was fully aware that the name of this operation is "logical or". There is also "bitwise or" which is a different thing. I started in languages that have them sharing the same operator, but they are introduced differently nevertheless. Neither is the same thing as "or as in the common sense".

Depending on how old the kids are, it may remove some ambiguity by using the longer name in teaching.

Doing that in language design may not sound clever. It would be something like extending the keyword to logical_or. Or you could use the C-style operator ||, and say its name is "logical or" and there isn't a shortened "or" mentioned anywhere.

Answer 10

If the problem were purely one of being confused about syntax, we could avoid the confusion by eliminating all implicit associativity and precedence in the infix operators (except perhaps for addition, subtraction, multiplication, division and perhaps exponents, if we have them).

Thus A == B or C would be a syntax error, requiring parentheses: either A == (B or C) or (A == B) or C.

However, even with the mandatory parentheses, students will still get tripped up and believe that A == (B or C) means (A == B) or (A == C).

Parentheses do not preclude distributive laws, whether real or imaginary. After all A * (B + C) is A * B + A * C!

Experienced programmers in C have a momentary lapse of reason and write something like if (state == (STATE_INITIAL || STATE_REQUEST_RECEIVED)) ...!!! This is a thing that happens; I've seen it in a code review more than once.

That particular expression pattern could be targeted by a code walker in your language, and subject to a diagnostic:

foo.lang:42:warning: X == (Y or Z) doesn't mean (X == Y) or (X == Z).

This will irks someone who knows what they are doing and wants that expression. That programmer wants to compare A with B, except when B is missing, in which case C is taken instead and compared with A.

One possibility would be to have a class of warnings that are targeted at neophytes. All learning materials would steer students toward operating the language in a way that the newbie warnings are turned on.

Ultimately, the way to reduce the semantic confusion would be to make a "Blub" languages. Don't have a clever or operator that returns one operand or the other. Have a pure Boolean or that yields true or false.

Then make it a type error to compare some X which is not Boolean to a Boolean value!

X == (Y or Z)     # error: comparing integer X with Boolean expression (Y or Z)

The idea of A or B or C yielding the leftmost one of A, B and C that is not falsy, and not evaluate the rest, comes from Lisp:

  (or A B C ...) ; yield A if that is true, else B if that is true, else ..

However, while it is that way in Common Lisp and related dialects, it was not so in Lisp 1, according to the 1960 manual.^* While Lisp 1 had N-ary AND and OR operators, which were short-circuiting, they yielded a strictly Boolean value. Doing anything like (EQL EXPR (OR A B)) in Lisp-1 would have meant that EXPR has a Boolean value, T or F, and that is being compared to the truth value of the OR. If EXPR contained, say, a number, the comparison would be false even if both A and B contained that number.

So before Python could borrow the Lisp or; first the original Lisp one had to be upgraded to the more clever semantics.

Under the new, clever or, you will sometimes get the expected behavior. If expr evaluates to 42 and so does a, then (eql expr (or a b)) will in that case behave consistently with the belief that expr is being separately compared with a and with b. The student will run the code and not notice the problem. The failing test case is something like (eql 42 (or 73 42)). Cases like (eql 42 (or nil 42)) and (eql 42 (or 42 nil)) do not reproduce the issue, and can confirm the wrong belief.

---

See page 48.