120 comments

[ 3.1 ms ] story [ 188 ms ] thread
I'm still looking to understand the difference between recursive definitions and inductive ones...
If you've looked at recursive functions then you should also look at recursive functions.
To not understand recursion, you must first... wait, I think that's all!
The jargon file entry for "tail recursion" read: "If you haven't grown tired of it already, see tail recursion."
Iteration is just recursion over a list/array/generator.
I don't know if that's a helpful framing?

    def sum(arr):
       total = 0
       for x in arr:
          total += x
       return total

and

    def sum(arr):
      if len(arr) == 0:
         return 0
      return arr[0] + sum(arr[1:])

both operate over an array but I'd call one of them recursive and the other iterative.
I think the closest short answer is that recursive definitions are just a special case of inductive ones for writing functions.
Do people talk about "inductive definitions"? I mostly only hear about induction in the context of proofs, in which case I'd say that an inductive proof/argument is one that uses the recursive structure of whatever it's about.

A recursive definition is: "natural numbers are 0 or a natural number + 1". An (abridged) inductive proof that uses the recursive nature of natural numbers: "the sum of all naturals up to n is n(n+1)/2: 0*(0+1)/2 = 0, and (n+1) + n(n+1)/2 = (2(n+1) + n(n+1))/2 = (2n + 2 + n^2 + n)/2 = (n^2 + 3n + 2)/2 = (n+1)(n+2)/2 = (n+1)((n+1)+1)/2"

But how do you know that induction is valid to use on natural numbers? It's because their definition/construction follows a set of rules that makes them inductive. Some languages like Coq make this explicit by providing an 'Inductive' keyword for defining inductive types.
> A recursive definition is: "natural numbers are 0 or a natural number + 1".

Note that this isn't enough by itself to support proof by induction.

Consider the definition of a list in Haskell which is analogous to the above definition of natural numbers: "lists are [] or a list with an element prepended".

Infinite lists satisfy this definition.

While I agree with the overall argument, reading this was a bit frustrating. The article made all kinds of interesting points and observations, but didn't really explain any of them. For example, I'd love to know what the difference between recursion and cyclicity is.
A recursive definition for something makes reference to that self-same definition, but it always has a base case to break the endless regress. Non-programming example: in grammar, a noun phrase might be defined as a noun, or a modifier and a noun phrase.
Infinite recursion (without a base case) is still recursion and, with tail call optimisation, can be useful in practice for the same reasons infinite loops can be useful.

I don't really see why the author doesn't see the joke as recursive. If it's about the lack of a base case they're misguided. Maybe the author is distinguishing between functions that recurse by calling themselves directly (direct recursion) and ones that call themselves via other functions (indirect recursion). Or maybe it's about recursive functions vs cyclic data structures. Or maybe it's something else as neither of those explains why the quoted joke is incorrect either.

You'd think in a section titled "Recursion vs Cyclicity" the author would make some effort to explain the difference, but apparently that matters less to them than calling things "dumb".

I don't even think it made interesting points, just "haha, who uses factorials and Fibonacci numbers?" and "I bet students don't like programming the same thing in different ways." (No? That part definitely interests me, but either way it depends on what would resonate with your student audience.)
Just let it speak for itself.
(comment deleted)
I agree with the overall sentiment. I taught for awhile and was happy with the approach I came up with. Students start out with the basics of learning how to loop through a list to transform some data. Once they have looping down, they are introduced to map and other built in methods. But, before using these methods, they are tasked with rewriting them using the looping they've learned.

Once they've "earned" the usage of the built in methods, they are tasked with rewriting them again, but this time without using any kind of looping. I give them a bit of time to think about how they may do this. Very few students get it but the plan is to live code it myself as an introduction to recursion. The task is still the same: to rewrite the map method. So the context for their intro to recursion is something they've become quite familiar with. It seems to have worked well.

> but this time without using any kind of looping.

See, to me, recursion is a kind of looping.

Yeah fair enough, I agree, I could have worded that differently. They couldn't use any of the looping they'd come to learn up to that point.
The tl;dr here is “just use HtDP” (which I agree with).
I would also include The Little Schemer and Learn You a Haskell For Great Good in the “further reading/references” section. They also explain recursion well.
Shriram Krishnamurthi, the author of this piece, happens to be also one of the authors of HtDP.
(comment deleted)
Where recursion really shines is when an iterative method would need an explicit stack, and a recursive one can use the call stack. Depth first search comes to mind.
However, since stack space is usually much smaller than heap space, if the problem size depends on external input and doesn’t have a sufficiently low limit, one either has to perform an appropriate size check beforehand or use an explicit stack after all.
They don’t mention “divide and conquer” at all, which I find a pretty compelling use of recursion even for non-inherently recursive problems. Writing a merge sort recursively is much easier and clearer than the nonrecursive variant.
The divide-and-conquer algorithm is a subset of what the post calls generative recursion, which requires a non-trivial amount of logic to get it right (even the simplest algorithm like binary search can be surprisingly difficult).
Ah, the post neglected to explain what they mean by generative recursion. The difficulties however are “just” in off-by-one errors and in the boundary cases of fixed-size integer types, which don’t get in the way if understanding the solution approach. You can easily understand it without having to understand recursive data structures. In that aspect, it shares the simplicity with fibonacci etc., but is more compelling than those because the alternative iterative solution is more difficult, compared to e.g. iterative fibonacci. The benefit of the recursive solution is more obvious.
I feel like I've read this before
The first mistake in teaching recursion is to make the first mistake in teaching recursion.
I see nothing wrong with either factorial or Fibonacci numbers.

The author argues that nobody uses or needs neither factorial nor Fibonacci numbers.

But that is not the point. If you are going to teach a concept it's best to use easy to understand examples. And factorial Fibonacci numbers are just that. I rather use factorial to teach recursion than QuickSort using Hoare partition algorithm.

And if it comes about teaching I think it is very important to so teach tail call recursion and to teach when to use recursion. Because it can be a waste to use recursion if an easy to implement iterative solution exists.

(comment deleted)
Agree. You could use the argument that argument against almost any teaching example. Like, when's the last time you ever needed to integrate x from 0 to 1? I've never needed to do that. Or when is the last time you needed to know how long a 1 kg ball will take to fall from 1 m? At least in my opinion, the way the author thinks recursion should be taught seems to needlessly complicate it.
I've done a few integrals of x from zero to one, they come up quite a bit if x is normalized.

Exception proves the rule, though ;-)

i think the problem is that most people try to follow the recursions in their head which is hard.

the biggest lesson regarding recursion is that if you're lucky enough to have your problem fit with tco (or have guarantees your problem is small enough), it's actually way simpler to both write and verify. write the base case, write the inductive step, translate to code, done. no hard reasoning about the code required because math. fact and fib demonstrate this nicely.

would prefer if programming languages offered a recursive decorator that would cause compilation or linting failures if tco isn't possible though.

>i think the problem is that most people try to follow the recursions in their head which is hard.

It's hard for computers, too as it can easy result in stack overflow if there are too many steps.

that's the obvious issue. another less intuitive one is that modern cpus and compilers are essentially optimized for iterative algorithms. state variables can be kept in registers, simd can be used for data that appears in contiguous arrays, caching is built on locality assumptions and data locality is preserved.

non-flattened recursive algorithms spread their state across linear memory with a full stack frame for every iteration. at the least this defeats gains from caches, at worst it defeats them entirely by filling them with junk.

but, if the problem is small enough it can be worth it for the correctness guarantees and increased simplicity.

trivial iterative algorithms are simpler, sure, but when things get more complicated, recursion can replace very hard to understand, test and debug iterative code with simple recursive definitions where guarantees of correctness and termination come for free.

If you go with a continuous passing style (which more often than not is relatively "easy") the compiler usually optimize it to replacing/reusing the frame rather than allocating a new one.
so it turns out that optimization is also tco (tail call optimization). that said, i was previously unfamiliar with continuous passing style, thanks!
This might be the root of the problem for the author:

>>> In principle, natural numbers are also recursive data: a number is either zero or the successor of another natural number. However, this way of thinking is not natural to students...

That's because students didn't learn math. "Back in my day," we learned mathematical induction sometime in middle school, and were re-introduced to it from time to time as a style of proof. When I finally got a chance to learn programming (in 1981), recursion was taught as something we already knew, being related to induction.

Which is closer to how it should be taught.

The factorial example is trivial and too familiar. It's easy for students to think "Oh, I know that" when in fact they don't get it at all.

Recursive operations on data structures are more likely to be unfamiliar. And you need something unfamiliar to demonstrate the point and also illustrate how it's likely to be used in practice.

Induction isn't recursion. Induction is why recursion works.

Unless you are assuming ZFC, which is much more comlplex then recursion itself, in second-order logic Induction is an axiom that we use to avoid recursion, by setting up the recursive step and then saying "so we don't have to execute it and look further"

Constructing the naturals is dynamic programming (building up), not recursion (breaking down).

Calculatiny addition using Peano axioms, that is recursion.

We hadn't gotten to those kinds of concepts in school by the time I learned recursion. The math that confounds programmers is much more basic.
Most of the time a sequence of numbers is recursively defined. Ex. Xn = Xn-1 + 2n

So a student might be familiar with it.

I would agree with the author that there are other, more useful examples of recursion. He gave a family tree example but for me ‘list all files on your hard disk’ would be a better example.

Tail call recursion is interesting, but that belongs in an FP course, which is usually taught long after the basics of programming.

I completely agree with this. The trouble also is that "real" recursive problems tend to be _very_ hard: an example from physics would be magnetohydrodynamics where a plasma moves in accordance to an applied external B field, causing a current to flow, which generates a B field that changes how the plasma flows...you get the idea.

Everyone knows that you don't _really_ want to try doing 123! using your "my first recursion" code in C. Similarly everyone knows that factorial gets pretty big pretty quickly – I remember finding out the limits of calculators in school by finding at which point x! went from "big number" to "error". It's pedagogically useful for those things and you can see (a) if your answer is right, compared to a BigNum library, and (b) how long it takes. There's a whole can of worms you can go down, from Sterling's approximation to a discussion about time complexity, ints and IEEE 754, and stack overflows. You could then go waffling about, e.g. Haskell and lazy evaluation and functional programming and so on.

Factorial isn't a good problem but it's common enough that students will have heard of it, deep enough to be interesting, and used so frequently as a programming example that if you see a recursive definition of factorial in another programming language you get that they're trying to teach you about recursion. It's a good teaching tool for all those reasons.

It was counterproductive for me when recursion was taught with Fibonacci and factorials. They were literally the opposite of easy to understand for me, I had absolutely no idea why those examples actually worked, so I just gave up and treated them like magic.

I think recursion should be taught with examples that would be easier for a student to write with recursion than without it. For example, listing all files in a directory.

Why would you think enumerating a directory is not magic?

Walking a tree / BFS is quite easy to implement iteratively, using a search queue. It's a common example for teaching Lisp.

Recursion is most natural in problems like implementing evaluation of an abstract syntax trer:

    eval(tree) = 
      apply(node(tree), 
            map(eval, leaves(tree))
           )
And since a tree is a graph, you can just use adjacency matrix and two loops. :)

O(n^2) but very simple.

That would require OS specific APIs which are distracting.
Python has other issues which make it less than ideal for a learning language. Really it's having to deal with an API at all that's distracting anyway.
(comment deleted)
The best summary I’ve heard and remembered was from my undergraduate data structures professor over 30 years ago. It has a nice rhyming structure: “every recursive function has a base case and a smaller caller.”
What does smaller caller mean in this case?
This probably is supposed to hint at a termination order. If a call to a recursive function terminates, then it it often (not always) possible to identify an ordering relation among the arguments of the call and the argument of the recursive call(s) within.

Though, there is also the Ackermann function... so for some recursive functions it is not be as easily seen why they would terminate.

> (not always)

Doesn't this work: consider the call graph of all possible argument lists (with an edge A->B if f(A) calls f(B)). If the function terminates, there are no cycles, so this is a DAG, which puts a partial order on the set of argument lists which strictly decreases with each call.

(I need sleep, Ackermann is actually very easy to show as decreasing using lexicographic order).

Yeah, if you know that the recursive function always terminates, then you know that each recursive call makes the set of remaining steps strictly smaller.

When you want to prove termination, then it is exactly what you need to show (that the remaining work does get smaller, no cycles).

You may find the Collatz conjecture interesting: https://en.wikipedia.org/wiki/Collatz_conjecture

It's an easy to understand loop, which can be written as a recursive function of one argument.

Even though the possible arguments are just the natural numbers, nobody knows how to identify a suitable ordering relation which decreases with each call.

I’ve always took it to mean the search space of the function needs to narrow to approach the base case so that it can terminate. In factorial this means subtracting 1, in tree searches it means picking one of the children, in parsing it means selecting a production that narrows the matches, etc.
> Towers of Hanoi

This also suffers from having a non-recursive solution, using bit-manipulation of a binary counter:

    max = 1 << no_of_discs;
    for (x = 1; x < max; x++)
        printf("move a disc from %d to %d\n", (x&x-1)%3, ((x|x-1)+1)%3);
Every recursive problem has a non-recursive solution.
True; but this one manages to avoid using an explicit stack.
(comment deleted)
> An attempt at a better answer might be that a problem is “inherently” recursive. However, that’s just a limitation of viewpoint. There’s nothing more inherently recursive than iterative about factorial. All the problems above can be expressed even more declaratively, as in a mathematical specification, that eliminates any implementation directive (e.g.: the greatest common divisor is the greatest, common, divisor: the definition of the problem says nothing about how to find it, and searching through all numbers to find divisors that are common and taking the largest one is no less valid a solution).

I always thought Euclid's algorithm had a pretty great runtime complexity, compared to "list all divisors, choose the greates common one". Is that not correct?

I think the point is that you can implement Euclid's algorithm (or any algorithm) iteratively as well as recursively. So there's nothing inherently iterative or recursive about any particular solution, let alone about any particular problem.
I have some comments on how (not) to teach it.

First of all, do not tell students to "start with the base case". The base case will be trivial and gives very little insight into how to solve the recursion problem. All the time they spend thinking about that is time spent not confronting the actual problem.

Beyond that, I think the way to teach recursion is to say "You cannot write the code until you can write a sentence in words describing the recursive solution. Your sentence must make reference to calling the procedure on a smaller sub-problem. Don't bother describing the base case in the sentence; it's boring - just fill it out when you write the code.".

So something like "The number of ways the robot can get to the last square is equal to the number of ways it can do it assuming it starts by going right plus the number of ways it can do it assuming it starts by going down".

I disagree with the reasoning for why factorial is a bad example. Factorial is very simple to explain as a mathematical function, requires very little prior knowledge. When asked why you aren't allowed to implement it as a loop, the response I was given was that you very simply define factorial in a self referential way (0! = 1 and n! = n * (n - 1)!), and is there any way you could implement the function in a way that clearly expresses that definition? At least for me this function was a fantastic intro to recursion
I agree factorial is very clearly defined with self-references but I think that the answers have no clear meaning and that's why its not an optimal way of teaching recursion. Teaching examples that have meaningful answers IMO allow for much better reasoning
agreed - it's the intuitiveness of the answer to the example that is so critical to building understanding. When you want to teach a method, it's much better if the student already knows what the result should be by a different method, so they can see how the new method is different than the one they know.
My favorite recursion example is multiplication. Multiplying two numbers is inherently recursive, even though we don't typically think of it that way. (Following is expressed in base 10 for clarity, but in base 2, the multiplications by powers of 10 are of course merely shifts so they don't cost anything.)

   68628933
 * 26973931

 =

   (6862 * 3931) * 10000
 + (2697 * 8933) * 10000
 + (6862 * 2697) * 10000 * 10000
 + (8933 * 3931)
And then we notice that (6862 * 3931) can be expressed as

     (68 * 31) * 100
 +   (39 * 62) * 100
 +   (68 * 39) * 100 * 100
 +   (62 * 31)
and so on for the 3 remaining products. And then each of these two-digit multiplications can in principle be subdivided into 4 1-digit multiplications.

The reason the algorithm is recursive is that we're really talking about polynomial multiplication, which is convolution. (Other kinds of multiplication (e.g. dot products) are not convolution and this discussion doesn't apply to them.)

The above is interesting and curious, but is it useful? Yes. Karatsuba multiplication [0] exploits this recursion and adds some clever algebra to reduce the total number of multiplication instructions needed and thus speed up bignum multiplication.

For even bigger numbers, the Schönhage–Strassen algorithm [1] goes even further and uses the Fourier transform to convert the overarching convolution operation to element-by-element (dot-product style) multiplication.

[0] https://en.wikipedia.org/wiki/Karatsuba_algorithm#Recursive_...

[1] https://en.wikipedia.org/wiki/Sch%C3%B6nhage%E2%80%93Strasse...

Exercise: Based on the above discussion, why should squaring an n-digit number be faster than multiplying 2 different n-digit numbers?
Even the basic schoolbook algorithms are recursive

    a * (10*b + c)
  = 10*(a*b) + (a*c)
In fact the usual method is even non-tail recursive, you build up a (literal!) stack of intermediate results which you sum back up at the end.
I think most schoolbook algorithms are tail recursive.

They're a bit hard to write down otherwise.

The usual method is iterative:

for digit in a, for digit in b, multiply digits and offset by sum of indices of digits. The sun results.

A simpler way multiplication is recursive is the way it is defined for natural numbers in mathematics:

  a × b = 0                if b = 0
  a × b = a + a × (b - 1)  if b ≠ 0
The fact that the recursive method for calculating factorials fails for quite small numbers is a good reason for teaching it.

Also computing the factorial is one of the first examples illustrating that recursion is useless in practice.

> The fact that the recursive method for calculating factorials fails for quite small numbers is a good reason for teaching it.

That variables can overflow is a valuable thing to teach, but it shouldn't be something the student bumps into during the lesson on recursion

The fact that the recursive method for calculating factorials fails for quite small numbers is a good reason for teaching it.

Also it's a good illustration of how useless recursion is in practice.

Iterative methods for calculating factorials also "fail for quite small numbers" in most common languages (basically, anything without built-in bignums).

The cause of failure in this case has nothing to do with recursion.

also you want to teach: why is iterative factorial faster than recursive? and , why do some (arbitrarily) large recursive factorials crash while the iterative equivalent just takes a long time ? (stack overflow)

IMHO we don't teach compsci kids enough about the costs of things - new vs. * for example

I didn't express myself well. That naive computations of factorials are useless for computing 50! is a useful lesson.

It's also useful to learn that despite the love of recursion among computer scientists, in practice it's barely if ever a good idea.

I think the family tree example would work better upside down, as you can easily state a problem that is obviously more intuitive to solve recursively (depth first search).

Start by defining a node object (a person) which contains a name and a list of their offspring (other nodes).

Find if the given starting node or any of their descendants have a certain first name, or some other property.

I've been a SWE for 12 years and has been teaching people how to code during my free time ever since I started coding (so more than 12 years of teaching).

I have taught around 30 students over the past 12 years who went from knowing nothing to getting a SWE job and from my limited dataset I have observed that:

1. Students who were taught for/while loops first has a hard time grasping recursion. I suspect it is because following the recursive callstack gets tricky and feels unintuitive. This inspired me to try a curriculum where I teach students recursion first and don't expose them for/while loops until they are prepping for interviews.

2. Students who were taught recursion first has no problem understanding for/while loops when they were exposed to it.

For people who are curious, I used to teach students at my local library but recently created a free online curriculum at: https://c0d3.com

Your site looks and reads fantastic; I can’t wait to give it a shot!
My background is in teaching CS1 courses in undergrad (as a TA).

There have been various studies that agree with you: teaching recursion before iteration seems to have benefits! From my experience, it seems to be extremely unpopular among teachers due to the claim that recursion is "useless in industry". My personal opinion is that learning recursion first is probably a step in the right direction, but I'm curious to hear your thoughts (in terms of their claims of the usefulness of recursion in industry).

> recursion is "useless in industry"

From my experience its true. I have never written recursive code in my 12 years of professional experience (primarily JS).

But during interviews, I have written recursive solutions many times to solve some of the harder problems.

IMO A major mistake books make is immediately attempting to visualize the call stack. This is important to be able to do. However, practitioners will simply imagine the code working on an arbitrary element and somewhat ignore the call stack (except perhaps for debugging).

I disagree that factorial is not "applied" enough -- no matter what problem you choose, it will be interesting to some students and not others. The entire purpose of recursion is not speed, but to more aptly fit your mental model -- resulting in simpler code.

In that light, I am also surprised that the article did not explain that recursion is essentially solving a problem in terms of a simpler problem, e.g. n! = n*(n-1)!, which eventually becomes the trivial case 0! = 1.

I agree you should start with the data structure. Give someone a recursive data structure and have them try to write a function that does stuff with it.
Yes, a binary tree and a simple function, like calculating the sum of all node values. That's a no-brainer. The funny thing is that when you present it the students first find it obvious (e.g. "sum = value of node + sum of left child + sum of right child, yes, easy, makes sense"), but then they start to think about it and then comes the "wait, why does that work???" phase, and it turns out they had not understand how procedure calls, arguments, and local variables really work up to that point.