Busy beaver numbers are interesting from a computational perspective. They verge in some ways on unknowable, but as this post shows, sometimes there is some knowability, if you will. I've always enjoyed this blog post about them https://www.scottaaronson.com/writings/bignumbers.html
I feel like I keep failing to wrap my mind around computability in the same way over and over.
Would the following statement be correct?
"There is a turing machine that cannot be proven whether it will terminate or not. For any n we can construct a proof that it will run longer than n. But we cannot construct a proof that for any n it will run longer than n. Due to this particular turing machine, BB(k) cannot be determined for k large enough to admit that turing machine."
So then if the BB sequence has finitely many entries. What does it mean that it grows faster than any computable sequence? A finite sequence can't have any asymptotic behavior right?
The BB sequence still has infinitely many entries. Any turing machine will either halt or not halt. For any N, there is a finite set of N-state turing machines which halt, and there is therefore a finite number of steps after which all halting N-state turing machines would halt.
There is a point where these values become uncomputable, but they still exist (for some definition of exist).
I am not sure I follow? But maybe I misunderstood your quote. It seems any given TM (And for a fixed N, there are only finite many), there are only the two option: it halts, or it doesn't halt. There is no third option?
Of course, the question whether you can proof that it doesn't halt, in ZF or ZFC, is a different question. (You can always proof that it halts if you know that it halts. You run it until it halts)
I'm not entirely sure about what I'm unsure about, but I think I struggle to accept that there is truth beyond the axioms. Like in my mind if the axioms can't prove whether it halts or not then that's that. There is no "but actually it doesn't halt". Or at least that's where my current thinking is, I think?
I do lack a lot of knowledge about the concept of interpretation, so maybe that's where my issue is?
All great questions. You're dancing around the edge of model theory here and the difference between truth and provability. It's probably past what can be explained well in a comment. Nevertheless, with omissions, simplifications, and my limited knowledge:
In logic, a model is a "set" of objects and all the true facts about those objects. For example, the integers and "1+1=2,1+2=3...". If a model deals with truth, then a "theory" is a mathematical structure that deals with provability. For example, the theory of groups contains an axiom "a+0=a" that applies to any a.
We say that a model "models" a theory when everything the theory proves is true in the model. The integers are a model of a group.
But, the integers mod 2 ({0,1}) are also a model of a group, but a very different one.
So, consider the statement "1+1≠0". The theory of groups isn't powerful enough to prove or disprove this proposition. It is a false statement in the integers, but a true statement of the mod2 integers. So there is a gap between what is true in a model, and what is provable in a theory.
(Look up both godel's completeness theorem and his incompleteness theorem for more insight into this relationship)
Back to turing machines, our most popular foundations of math is a theory named ZFC. With this theory we can prove many things, but there are also many things that can't be proven or disproven. ZFC lets us prove that "BB(500) is an integer greater than 5". But it doesn't let us prove that "BB(500)=174936...35" (pretend that's a concrete number). In fact, there probably is a model of ZFC where it is true that BB(500)=174936...35, and there is also probably a model of ZFC where BB(500)=284729...86.
Axioms define what is provable, but godel showed that there will always be more true things than there are provable things (if your theory is finite-ish). Most concrete questions about the BB function will never be provable.
Where did you get that the BB sequence would have finitely many entries? You can choose any integer n for BB(n), and there are infinitely many integers.
As others have said, the sequence is infinite, though unknowable. Even if math can't say what the precise value is, it can still prove things about the shape of the sequence.
As a simple example, we know that BB(1000) <= BB(1001) (since you can encode the longest running 1000-TM with an extra, unused state as a 1001-TM) even though we will never know either of those values.
The argument that it grows faster than any computable function is similar in that it proves things without knowing concrete numbers. (I'd have to look up the details of the proof to say more)
You should be able to prove it by showing that the existence of a faster-growing computable function lets you solve the halting problem.
Assume f(x) is computable and grows faster than BB(x). We can create a computable g(x) where g(x) > BB(x) for all x, by setting g(x) = f(x) + C for some suitable constant.
Now we create a computer program that does the following. 1. Get the size of the input program and compute g(n). 2. Run the input program for g(n) steps. 3. If it stopped, return true. If it's still running, return false.
This solves the halting problem. Since a computer program can't solve the halting problem, this program can't exist. Everything except "compute g(n)" is straightforward, so there must be no such function g, and no such function f.
Essentially yes, but it's simpler and stronger than that: Any oracle that can output BB(n) given n, can be harnessed to solve the halting problem. Exercise! Try to show that: For any computable integer sequence U, BB(n) > U(n) for all but finitely many exceptions.
Every BB(n) is trivially computable, since any integer is computable. So for every BB(n), there is a computable sequence that grows faster than the BB numbers through n (annoyingly, given the word "grows", this can just be the constant sequence of BB(n) + 1).
So for any given computable number, the fact that it's computable gives no intrinsic reason to believe it's smaller than BB(n) for any particular n.
No, high enough busy beaver numbers are not computable. They can be defined but there's no way to know what they are.
That is because if you could compute BB(n) for arbitrary n, you could run a Turing machine for BB(n)+1 steps and if it hasn't stopped yet then you know that the input corresponds to a machine that never halts. So it's impossible to compute BB(n) for arbitrary n, because doing so is equivalent to the halting problem.
Even for very small n you can create Turing machines where it's incredibly hard to prove that they halt: for n=25 you can create a Turing machine that stops only if the Goldbach conjecture is false. Though that isn't enough to prove it's not computable.
What I mean is that any given integer is trivially computable.
The busy beaver sequence is not computable (I am aware that somewhere around BB(700), you've encoded the consistency of ZFC). You cannot necessarily compute that BB(n) is a given number. But you can compute larger numbers. There is no such thing as integer that is too large to be computable.
I did slightly fuck up--the English language "every BB(n) is computable" is a bit ambiguous. What I should say is: "for all n, BB(n) is trivially computable, since it is a single integer".
BB(5) is also computable in the intuitive sense, because we proved its value: 47,176,870. In this sense BB(5) is just as computable as Graham's number. However there is likely a Busy Beaver number >5 that is intuitively uncomputable, such that there is no non-cheating algorithm which computes it. A cheating algorithm would be one that requires the knowledge of the result in advance to write the algorithm, in which case we wouldn't be able to come up with the algorithm in the first place, and we wouldn't know that the result is correct.
There is also a proof that the function BB(n) (with the free argument n) is uncomputable. However, this alone doesn't mean that e.g. BB(6) (a function with no free parameter) is or isn't intuitively computable. Though if you allow cheating, which is the case for the conventional academic definition of "computability", every particular constant function (that is, without a free parameter) would be considered "computable". Though that definition doesn't mean it is intuitively computable, because if you cheat, you aren't really computing something something, you are just reporting something without having computed it.
It's code golf played with Turing machines. The game is: Make the smallest Turing machine whose runtime is bigger than Graham's number, but still terminates. The record right now is 10 states. At least 6 states are needed because any Turing machine with at most 5 states, that finishes running, has a runtime of at most 47,176,870, which is less than Graham's number. In summary:
It's not "relevant" in any sense, but I think many computer folks are fascinated by busy beaver numbers because it's fairly easy what the numbers mean - it's just a Turing machine with no icky math.
That said, functionally, they're more or less a restatement of the halting problem, essentially "if the halting problem was a number". Also see Godel's incompleteness. A bit mind-blowing that you have numbers you can't "know" (i.e., compute or verify), but that's about it.
People have been working on them for fun, with no chance of any of this being applicable to anything, so throwing AI at it to produce new results seems a bit tone-deaf to me.
I don't think that's right? The busy beaver numbers are all knowable, and we're figuring out what they are one-by-one. There's no single computation we can run which will compute all of them, but that doesn't mean they are unknowable or uncomputable. It just means you need more computational machinery (like, more algorithms and strategies) for each additional number in the sequence that you compute.
I think the idea here is: how do you write as many 1s as you can - without going into an infinite loop. You gotta stop. You only have 10 states in a state machine to "program" with.
That's BB(10).
The naive way to solve this is to generate all possible 10 state state-machines and see what they produce - take the biggest one.
Except many of those will go into an infinite loop - and you cannot easily detect that (or detect it at all for arbitrary n - see also: halting problem) so you go into an infinite loop too.
Too many requests
You have exceeded a secondary rate limit.
Please wait a few minutes before you try again;
in some cases this may take up to an hour.
Signing in may provide a higher rate limit if you are not already signed in.
For more on scraping GitHub and how it may affect your rights, please review our Terms of Service.
Ironic how now normal browsing of GitHub and this AI assisted proof is marked as scraping.
One problem with LLMs in academic contexts is that because you're being paid by the token, it's resulting in extremely short papers that don't adequately explain the context. It's the same problem with long-standing short papers.
I'm wondering - have you asked an LLM to flesh-out the paper for you? Or walk you through the details?
I've been hearing stories of mathematicians who are developing a new found appreciation for LLMs by having them explain the other LLM's paper. I guess the paper has the necessary details but possibly not the best exposition.
And nowadays LLMs are the experts. So. It will explain it to you if you ask. And it does a very nice job. You'll probably need to lose the attitude, tho.
The cost of the tokens used to output a typical academic paper length document is on the order of cents, even for frontier models. The vast majority of the cost is in the discussion leading up to that final output - there's no reason someone would try to produce a shorter paper just to save tokens.
Graham's Number is notable for being one of the first, being an actual relevant bound in an unrelated proof (not just the output of a function), of teaching people the Up Arrow Notation.
To me, BB(10), TREE(3) etc. being so large is only minorly more impressive than GG(1) where I define I define GG(x) as "Graham's Number + X"
I don’t think such a bound in itself is interesting, unless the proof reveals some strategy to prove more. There are infinitely potential bounds to BB(n), why is this particular case BB(10) > G useful for understanding fast growing functions?
45 comments
[ 7.0 ms ] story [ 94.2 ms ] threadWould the following statement be correct?
"There is a turing machine that cannot be proven whether it will terminate or not. For any n we can construct a proof that it will run longer than n. But we cannot construct a proof that for any n it will run longer than n. Due to this particular turing machine, BB(k) cannot be determined for k large enough to admit that turing machine."
https://scottaaronson.blog/?p=2725
There is a point where these values become uncomputable, but they still exist (for some definition of exist).
I guess this is the crux of the issue for me and what I'm struggling with. This does not seem obvious to me.
I do lack a lot of knowledge about the concept of interpretation, so maybe that's where my issue is?
In logic, a model is a "set" of objects and all the true facts about those objects. For example, the integers and "1+1=2,1+2=3...". If a model deals with truth, then a "theory" is a mathematical structure that deals with provability. For example, the theory of groups contains an axiom "a+0=a" that applies to any a.
We say that a model "models" a theory when everything the theory proves is true in the model. The integers are a model of a group.
But, the integers mod 2 ({0,1}) are also a model of a group, but a very different one.
So, consider the statement "1+1≠0". The theory of groups isn't powerful enough to prove or disprove this proposition. It is a false statement in the integers, but a true statement of the mod2 integers. So there is a gap between what is true in a model, and what is provable in a theory.
(Look up both godel's completeness theorem and his incompleteness theorem for more insight into this relationship)
Back to turing machines, our most popular foundations of math is a theory named ZFC. With this theory we can prove many things, but there are also many things that can't be proven or disproven. ZFC lets us prove that "BB(500) is an integer greater than 5". But it doesn't let us prove that "BB(500)=174936...35" (pretend that's a concrete number). In fact, there probably is a model of ZFC where it is true that BB(500)=174936...35, and there is also probably a model of ZFC where BB(500)=284729...86.
Axioms define what is provable, but godel showed that there will always be more true things than there are provable things (if your theory is finite-ish). Most concrete questions about the BB function will never be provable.
As a simple example, we know that BB(1000) <= BB(1001) (since you can encode the longest running 1000-TM with an extra, unused state as a 1001-TM) even though we will never know either of those values.
The argument that it grows faster than any computable function is similar in that it proves things without knowing concrete numbers. (I'd have to look up the details of the proof to say more)
Assume f(x) is computable and grows faster than BB(x). We can create a computable g(x) where g(x) > BB(x) for all x, by setting g(x) = f(x) + C for some suitable constant.
Now we create a computer program that does the following. 1. Get the size of the input program and compute g(n). 2. Run the input program for g(n) steps. 3. If it stopped, return true. If it's still running, return false.
This solves the halting problem. Since a computer program can't solve the halting problem, this program can't exist. Everything except "compute g(n)" is straightforward, so there must be no such function g, and no such function f.
Every BB(n) is trivially computable, since any integer is computable. So for every BB(n), there is a computable sequence that grows faster than the BB numbers through n (annoyingly, given the word "grows", this can just be the constant sequence of BB(n) + 1).
So for any given computable number, the fact that it's computable gives no intrinsic reason to believe it's smaller than BB(n) for any particular n.
That is because if you could compute BB(n) for arbitrary n, you could run a Turing machine for BB(n)+1 steps and if it hasn't stopped yet then you know that the input corresponds to a machine that never halts. So it's impossible to compute BB(n) for arbitrary n, because doing so is equivalent to the halting problem.
Even for very small n you can create Turing machines where it's incredibly hard to prove that they halt: for n=25 you can create a Turing machine that stops only if the Goldbach conjecture is false. Though that isn't enough to prove it's not computable.
The busy beaver sequence is not computable (I am aware that somewhere around BB(700), you've encoded the consistency of ZFC). You cannot necessarily compute that BB(n) is a given number. But you can compute larger numbers. There is no such thing as integer that is too large to be computable.
I did slightly fuck up--the English language "every BB(n) is computable" is a bit ambiguous. What I should say is: "for all n, BB(n) is trivially computable, since it is a single integer".
Graham's number is not a sequence, so it doesn't matter that the BB sequence grows faster than any computable sequence.
There is also a proof that the function BB(n) (with the free argument n) is uncomputable. However, this alone doesn't mean that e.g. BB(6) (a function with no free parameter) is or isn't intuitively computable. Though if you allow cheating, which is the case for the conventional academic definition of "computability", every particular constant function (that is, without a free parameter) would be considered "computable". Though that definition doesn't mean it is intuitively computable, because if you cheat, you aren't really computing something something, you are just reporting something without having computed it.
That said, functionally, they're more or less a restatement of the halting problem, essentially "if the halting problem was a number". Also see Godel's incompleteness. A bit mind-blowing that you have numbers you can't "know" (i.e., compute or verify), but that's about it.
People have been working on them for fun, with no chance of any of this being applicable to anything, so throwing AI at it to produce new results seems a bit tone-deaf to me.
I've had a lot of success answering questions like this by asking an LLM. It turns out they understand each other and do a great job explaining.
That's BB(10).
The naive way to solve this is to generate all possible 10 state state-machines and see what they produce - take the biggest one.
Except many of those will go into an infinite loop - and you cannot easily detect that (or detect it at all for arbitrary n - see also: halting problem) so you go into an infinite loop too.
Computation theory is fun.
Too many requests You have exceeded a secondary rate limit.
Please wait a few minutes before you try again; in some cases this may take up to an hour. Signing in may provide a higher rate limit if you are not already signed in.
For more on scraping GitHub and how it may affect your rights, please review our Terms of Service.
Ironic how now normal browsing of GitHub and this AI assisted proof is marked as scraping.
And this is a fine example of this.
I've been hearing stories of mathematicians who are developing a new found appreciation for LLMs by having them explain the other LLM's paper. I guess the paper has the necessary details but possibly not the best exposition.
How do I know when I read Claude's interpretation of the paper that it's not hallucinating.
And nowadays LLMs are the experts. So. It will explain it to you if you ask. And it does a very nice job. You'll probably need to lose the attitude, tho.
Consider TREE(3) -> https://www.youtube.com/watch?v=3P6DWAwwViU
Graham's Number is notable for being one of the first, being an actual relevant bound in an unrelated proof (not just the output of a function), of teaching people the Up Arrow Notation.
To me, BB(10), TREE(3) etc. being so large is only minorly more impressive than GG(1) where I define I define GG(x) as "Graham's Number + X"
I'd call it absolutely mind boggling, actually.