HN Explorer

Ask HN: How can GPT-4 reason?

2 points by eternalvoyage ↗ HN
Hello.

I just watched the GPT-4 Developer livestream. When GPT was reasoning about the tax problem, I got very confused. If this model predicts what word comes next (I know its much much more complicated), how can it understand and reason about a problem?

Is it a hand picked problem which it can solve accurately and will fail in future when given similar tasks?

Or is there some other module which gives GPT its reasoning ability?

Or is it some sort of emergent property of a system which predicts next word and has been trained on a huge amount of data?

I know they haven't released much in the paper, but I want to know if anyone has any theories about this.

Also, if it can understand and reason about a given block of text, why not give it description of a already solved but very complicated math / physics problem and ask it to solve that. If it can successfully solve that, we can try with an unsolved problem.

3 comments

[ 4.0 ms ] story [ 19.3 ms ] thread
gregjor
It doesn’t reason. People infer reasoning ability because that’s the only model we have in our brains for something that appears to communicate intelligently.

Stephen Wolfram wrote a good paper about how ChatGPT works and why it’s not reasoning about anything.

pfortuny
Look, you only have to ask chatgpt something totally nonsensical like

“Show me how a martingale converges to the zariski measure on a topos”

To see that it has no clue what it is “talking” about AND does not know that does not know.

Of course, you need a topic on which there is sacant literature.

eternalvoyage
I understand what you are saying, and I know that chat GPT generates nonsense on things it doesn't know and also does not respond that it doesn't know.

But let's consider that we feed GPT a blob of text, just like in the demo. Then ask it to solve a problem based on it and tell it to also show us the reasoning steps. If it just generates some meaningful text but does not solve the problem then we are sure it can't reason. But my question is how can it show me the steps? If it can't reason, where is the reasoning coming from?

I am trying to compare this with a human student. Let's say I gave a student the tax codes and nothing else. Then I give that student a tax calculation problem which involves knowing that the problem contains general tax calculation steps and also some special rules which are embedded somewhere in the tax code. When the student comes up to me with a correct answer, I don't immediately know that the student understands tax codes. To really know, I have to ask him how did you come up with the solution. If he can show me the steps then I will know he understood the tax codes. Beyond that there is no way to look into his brain and confirm that he has understood the problem.

So if GPT-4 can do the same, why would I not assume it has reasoning abilities? Maybe the process by which it comes up with the reasoning steps is different from us, but they are nonetheless reasoning steps.

I am not arguing that GPT can reason. I personally don't believe that. I am trying to understand how it can generate reasoning steps if it can not reason.