The paper was explicitly studying Java. I suspect performing the same exercise against OCaml or any other language would give you a very different list of very real problems.
I'm surprised mismatching parenthesis was a common mistake in 2016 given the ubiquity of IDEs and text editors that should catch a syntax error like that.
== vs .equals() is pretty understandable considering that it's Java specific, and would be valid in most(?) other languages
While == vs .equals() is a Java-specific issue, other languages suffer similar (and often more hidden) equality problems.
In C, you can have value equality or pointer equality. Then, you have the whole "either write your own character-by-character comparison or learn which libraries to include to compare strings". There's also the detail of remembering what strcmp() returns.
I might be wrong since Python is not my daily driver, but "a is b" and "a in b" and "a == b" and "a == b and type(a) == type(b)" are all distinct from one another.
The reason people confuse "==" and ".equals" in Java is because they are confusing. In general, Java doesn't use the concept of pointers, and yet, one of its main operators does.
That language feature is simply broken, and this is really not the same situation of confusing "a == b" with "a in b" or "type(a) == type(b)". One can say that comparison with None is Python is broken too, but the situation is much better because you get an error if you make a mistake, instead of your program misbehaving.
Javascript, by the way, is way worse than Java here. In general the language does not differentiate a number from a string with a numeric value (something I imagine it copied from Perl), yet a lot of features do.
In my experience IDE sucks for paranthensis/quote their autocompletion is awful because it only works well when you write new code not when you edit existing code..
And in VScode it isn't even possible to truly disable it!
It's curious the article didn't propose solutions.
> Confusing short-circuit logical operators
> Using == instead of .equals to compare strings.
These problems are more typo/syntax-based rather than logic-based.
> Ignoring the return value from a non-void method.
This is more logic-based but also something that IDEs or other static-analysis tooling can usually spot (I wish java had `[[nodiscard]]` but alas).
You can certainly say that Java's syntax is an additional layer of complexity to beginners, and this syntax can make it hard to see the forest for the trees.
But any language or learning environment is going to have its quirks. I wonder if the real problem here is trying to teach programming in a total vacuum. I.e., not also teaching how to program using a competent editing environment and teaching how to use its feedback to solve problems.
I taught myself PHP at 13 using only Windows Notepad uploading to the server and refreshing every change. The concepts of an IDE, a syntax-aware editor, or a local development environment never occurred to me despite having read plenty of examples and explanations.
I don't know if the teaching/discoverability situation is any better in general now, but in my teaching I spend the first lesson showing how to type, do editor things, and look at feedback in VSCode before I explain even the first thing about syntax or programming. I would hope that teachers of beginners would show some tooling that gives students direct and actionable feedback.
> These problems are more typo/syntax-based rather than logic-based.
This!
I was hoping for a list of logic and organization / structure errors.
On the idea of editor feedback, one thing I find frustrating is that hardly anyone teaches how to read error messages and stack traces. That’s the most important thing because as a beginner you make tons of errors and you see them a lot! Books and tutorials don’t show any errors, so when you inevitably make one it’s hard to figure out what exactly you did wrong.
This is novice programmers. They aren't writing big enough programs for organization/structure errors to matter much. They could have logic errors, but this kind of stuff is the fluff you have to get out of the way before you can start showing them the logic errors.
There's kind of a sequence:
- You didn't say what you meant (this kind of error)
- You said what you meant, but you meant the wrong thing (logic errors)
- You said the right thing, but in the wrong place (organization/structure).
With a novice, you might need to work through them in approximately that order.
I’m not sure what “novice” means. I certainly know when I was beginning I glossed over stack traces because they were a giant information dump and I didn’t know how to read them. They tell you exactly what’s happened, the call, what line it broke, what the local variables are, how it got there, etc. But it’s like reading a foreign language and is intimidating for a beginner.
IDEs help a lot but once you know how to read a stack trace it’s easy to understand if you made a typo:
AttributeError: Person object type has no attribute “helo”
…or have a bug:
AttributeError: None type has no attribute “hello”
It took me 15 years to convince myself I was no longer a novice. I've been at it for 40 years now, and I still don't think I'm anything like a master of the craft, nor ever will be.
> These problems are more typo/syntax-based rather than logic-based.
Interestingly enough, syntax-based mistakes are often the first hurdles students encounter in programming, as described in Altadmri's paper. They also go on to observe that these mistakes do subside as the semester progresses as students "get used" to syntax, while logic-based errors will persist.
While editors can provide some benefit to these issues, they can also be rather complex; often designed for industry rather than education. However these issues persist even with BlueJ, the IDE used in Altadmri's paper, which is an education-focused development environment.
My current research on the topic encourages the use of lower-level exercises (even typing exercises) with CS examples to help students get familiar with coding's syntactic structures without problem solving [1]. As a result, students are struggling on correctly solving the coding problem while struggling to correctly implement the syntax necessary. My recommendation is to rather provide additional practice opportunities that isolate these lower level skills so students aren't trying to problem solving AND refine their skills at the same time. In addition, these lower level exercises can provide students with additional examples that can be more beneficial that simply showing them code snippets [2]. These examples can help serve as 'templates' that students can refer to when they encounter similar problems.
The typing exercises remind me of 80s/90s computer magazines having basic/asm code for interesting programs or games. You had to retype them to run them.
It was slightly before my time, but I suspect a large number of people got through the "learn the tooling" stage simply by retyping, looking for the mis-type when they got it wrong, and learning how to understand the feedback.
The natural next thing to do is to try to change some values and see what happens. All of a sudden you're learning how the whole program and environment work together and can always go back to a known good state by retyping what the paper says.
> The typing exercises remind me of 80s/90s computer magazines
YUP! I've been collecting the old Usbourne and Micro Adventure books and typically bring them along when I give presentations to say "This is how they did it in the old days! Let's do it again!" I then compare it to other "technical" skills like playing an instrument, acting, and martial arts. Each one has a "you gotta memorize/drill this" phase that is imperative to higher level performance, creativity, or problem solving.
> The natural next thing to do is to try to change some values and see what happens.
There is some work out of the UK known as PRIMM that uses this as one of their initial steps [1]. I spoke with Sue Sentance at SIGCSE in 2017 when she presented her studies and it sounds like another cool way to present coding practice without problem solving. John Edwards out of Utah is also doing "simple syntax exercises" [2] for his style of drilling where you have students do 20-30 exercises on a concept without very minor changes like "make the loop go 3 times; now 4; now go backwards from 4 to 0; etc.".
>> Ignoring the return value from a non-void method.
Beginners might also write a method/function with a return parameter they don't care about (or that they sometimes don't care about) but that has some desirable side-effect.
A common issue I've seen is more the return is replaced with a print statement. As a result, their code "appears" to working because they're still seeing the right values appear in their console when they run the code. If the function's returned value doesn't get used somewhere else, then they don't see where the issue is until points are deducted.
> > Confusing short-circuit logical operators > Using == instead of .equals to compare strings.
> These problems are more typo/syntax-based rather than logic-based.
Sure, but you can also see why it could take absolutely ages for a novice to work out what's wrong, right? The bitwise operators will work for evaluating logic like this in some cases, but not all. And the student knows it's something like "&" for "AND" and "|" for "OR". Could be a really tricky bug if you've been introduced to logical operations, but not bitwise ones, and don't therefore realise they could exist & be confused.
To me, the more interesting thing about this article is that professors are only moderately correct about what mistakes their students typically make. This despite the fact that we might expect professors to be experts on what their students have a hard time understanding, and what mistakes often flow out of those misunderstandings.
The paper does not say that professors are only moderately correct about what mistakes their students typically make, just that their personal experience does not align with the results of the cohort used in the study.
A professor may be perfectly correct at a local level but globally only moderately correct. The contextual knowledge of the professor is not acknowledged in this study (curriculum, course level, teaching style, problem set, student background, lecture notes, textbook choice, compiler, IDE) nor are the professors own students who may only come to the professor or TA with a certain subset of problems etc.
One big mistake that I would frequently run into as a novice is not using source control, or at least saving versions of my work. I would start making a complicated change, end up breaking some fundamental part of the program, and wouldn't have any way of going back to a version that worked since I would always just save in the same file. Things like version control might seem like a chore to someone who has not had their time adequately burned by their own mistakes.
> Things like version control might seem like a chore to someone who has not had their time adequately burned by their own mistakes.
Haha, that is true of advice in general. People generally do not accept advice until they have enough experience to recognize that the advice would have saved them a lot of grief:
Year 0 - what's a buffer overflow?
Year 1 - I'm too smart to be one of those noobs who have buffer overflows. I don't need X that prevents them
Year 5 - wow, that's a great idea for preventing buffer overflows! I wish I knew about X years ago
1) Version control adds ANOTHER concept to students that can be difficult to grasp/appreciate. Adding it in is great from a practicality standpoint but increases the cognitive load of students with ANOTHER new technology.
2) Sometimes, the code just hits a point where its no longer salvageable. It sucks to tell a student they should start over, but often times wiping the slate clean after making a mess can help consolidate their thought process.
I think the most important mistake is writing needlessly complicated algorithms. I.e., creating more auxiliary variables than needed. Also 'if (condition == true)' tends to happen quite a lot.
When I started programming, the first mistake that I have encountered is that I did ignore the spacing rules in Python, thus creating errors even though the code was correct.
34 comments
[ 3.8 ms ] story [ 100 ms ] threadEdit: Found it (https://dl.acm.org/doi/pdf/10.1145/3287324.3287381)
== vs .equals() is pretty understandable considering that it's Java specific, and would be valid in most(?) other languages
The main reason I find myself second guessing this one is if you get used to the convention of other languages and then switch over to java.
In C, you can have value equality or pointer equality. Then, you have the whole "either write your own character-by-character comparison or learn which libraries to include to compare strings". There's also the detail of remembering what strcmp() returns.
I might be wrong since Python is not my daily driver, but "a is b" and "a in b" and "a == b" and "a == b and type(a) == type(b)" are all distinct from one another.
Javascript has strict equality and equality.
That language feature is simply broken, and this is really not the same situation of confusing "a == b" with "a in b" or "type(a) == type(b)". One can say that comparison with None is Python is broken too, but the situation is much better because you get an error if you make a mistake, instead of your program misbehaving.
Javascript, by the way, is way worse than Java here. In general the language does not differentiate a number from a string with a numeric value (something I imagine it copied from Perl), yet a lot of features do.
And in VScode it isn't even possible to truly disable it!
> Confusing short-circuit logical operators > Using == instead of .equals to compare strings.
These problems are more typo/syntax-based rather than logic-based.
> Ignoring the return value from a non-void method.
This is more logic-based but also something that IDEs or other static-analysis tooling can usually spot (I wish java had `[[nodiscard]]` but alas).
You can certainly say that Java's syntax is an additional layer of complexity to beginners, and this syntax can make it hard to see the forest for the trees.
But any language or learning environment is going to have its quirks. I wonder if the real problem here is trying to teach programming in a total vacuum. I.e., not also teaching how to program using a competent editing environment and teaching how to use its feedback to solve problems.
I taught myself PHP at 13 using only Windows Notepad uploading to the server and refreshing every change. The concepts of an IDE, a syntax-aware editor, or a local development environment never occurred to me despite having read plenty of examples and explanations.
I don't know if the teaching/discoverability situation is any better in general now, but in my teaching I spend the first lesson showing how to type, do editor things, and look at feedback in VSCode before I explain even the first thing about syntax or programming. I would hope that teachers of beginners would show some tooling that gives students direct and actionable feedback.
I was hoping for a list of logic and organization / structure errors.
On the idea of editor feedback, one thing I find frustrating is that hardly anyone teaches how to read error messages and stack traces. That’s the most important thing because as a beginner you make tons of errors and you see them a lot! Books and tutorials don’t show any errors, so when you inevitably make one it’s hard to figure out what exactly you did wrong.
There's kind of a sequence:
With a novice, you might need to work through them in approximately that order.IDEs help a lot but once you know how to read a stack trace it’s easy to understand if you made a typo:
…or have a bug:It took me 15 years to convince myself I was no longer a novice. I've been at it for 40 years now, and I still don't think I'm anything like a master of the craft, nor ever will be.
Interestingly enough, syntax-based mistakes are often the first hurdles students encounter in programming, as described in Altadmri's paper. They also go on to observe that these mistakes do subside as the semester progresses as students "get used" to syntax, while logic-based errors will persist.
While editors can provide some benefit to these issues, they can also be rather complex; often designed for industry rather than education. However these issues persist even with BlueJ, the IDE used in Altadmri's paper, which is an education-focused development environment.
My current research on the topic encourages the use of lower-level exercises (even typing exercises) with CS examples to help students get familiar with coding's syntactic structures without problem solving [1]. As a result, students are struggling on correctly solving the coding problem while struggling to correctly implement the syntax necessary. My recommendation is to rather provide additional practice opportunities that isolate these lower level skills so students aren't trying to problem solving AND refine their skills at the same time. In addition, these lower level exercises can provide students with additional examples that can be more beneficial that simply showing them code snippets [2]. These examples can help serve as 'templates' that students can refer to when they encounter similar problems.
[1] https://dl.acm.org/doi/pdf/10.1145/3373165.3373177
[2] https://educationaldatamining.org/EDM2021/virtual/static/pdf...
It was slightly before my time, but I suspect a large number of people got through the "learn the tooling" stage simply by retyping, looking for the mis-type when they got it wrong, and learning how to understand the feedback.
The natural next thing to do is to try to change some values and see what happens. All of a sudden you're learning how the whole program and environment work together and can always go back to a known good state by retyping what the paper says.
YUP! I've been collecting the old Usbourne and Micro Adventure books and typically bring them along when I give presentations to say "This is how they did it in the old days! Let's do it again!" I then compare it to other "technical" skills like playing an instrument, acting, and martial arts. Each one has a "you gotta memorize/drill this" phase that is imperative to higher level performance, creativity, or problem solving.
> The natural next thing to do is to try to change some values and see what happens.
There is some work out of the UK known as PRIMM that uses this as one of their initial steps [1]. I spoke with Sue Sentance at SIGCSE in 2017 when she presented her studies and it sounds like another cool way to present coding practice without problem solving. John Edwards out of Utah is also doing "simple syntax exercises" [2] for his style of drilling where you have students do 20-30 exercises on a concept without very minor changes like "make the loop go 3 times; now 4; now go backwards from 4 to 0; etc.".
[1] https://blogs.kcl.ac.uk/cser/2017/09/01/primm-a-structured-a...
[2] https://dl.acm.org/doi/pdf/10.1145/3372782.3406259
Try Google's Error Prone:
https://errorprone.info/bugpattern/CheckReturnValue
Or @CheckResult in Android-flavor Java
Beginners might also write a method/function with a return parameter they don't care about (or that they sometimes don't care about) but that has some desirable side-effect.
> These problems are more typo/syntax-based rather than logic-based.
Sure, but you can also see why it could take absolutely ages for a novice to work out what's wrong, right? The bitwise operators will work for evaluating logic like this in some cases, but not all. And the student knows it's something like "&" for "AND" and "|" for "OR". Could be a really tricky bug if you've been introduced to logical operations, but not bitwise ones, and don't therefore realise they could exist & be confused.
That's an atrocity.
A professor may be perfectly correct at a local level but globally only moderately correct. The contextual knowledge of the professor is not acknowledged in this study (curriculum, course level, teaching style, problem set, student background, lecture notes, textbook choice, compiler, IDE) nor are the professors own students who may only come to the professor or TA with a certain subset of problems etc.
Haha, that is true of advice in general. People generally do not accept advice until they have enough experience to recognize that the advice would have saved them a lot of grief:
Year 0 - what's a buffer overflow?
Year 1 - I'm too smart to be one of those noobs who have buffer overflows. I don't need X that prevents them
Year 5 - wow, that's a great idea for preventing buffer overflows! I wish I knew about X years ago
1) Version control adds ANOTHER concept to students that can be difficult to grasp/appreciate. Adding it in is great from a practicality standpoint but increases the cognitive load of students with ANOTHER new technology.
2) Sometimes, the code just hits a point where its no longer salvageable. It sucks to tell a student they should start over, but often times wiping the slate clean after making a mess can help consolidate their thought process.