It's a collection of arguments for the impossibility of super-human intelligence, rewritten to refer to size instead. If one is absurd, so is the other.
> Although the technical details from this point onward are unfortunately
too dense to include in a general-audience essay of this sort, assuming as they
do familiarity with constructive non-standard analysis, it suffices to say that
supersized machines cannot be made (Figure 2).
Brilliant. This is exactly how "Godel's Theorem or Quantum Mechanics therefore I'm right" arguments always go :)
Is anyone actually arguing that a super-intelligent AI is, like, physically impossible or something? That seems like a very dumb position to hold, it would be necessary (but not sufficient) to believe that human brains are somehow following alternative laws of physics (aka, magic).
The room to argue, I think, is in how close we are to making them and whether things like DNN’s are the path or a dead end.
In almost any HN thread that touches on the subject someone manages to express this sort of view. E.g. from the most recent thread I read: https://news.ycombinator.com/item?id=40686920 (apologies for singling out the post, it was impossible to give an example without doing singling out something-- I hope you'll read it in the sense it was intended: an example of a common pattern).
It's even more common in other venues.
That said, from what I've seen the posted article is often used to justify hysteria about AI. For some people the thing protecting them from a meltdown is just a view that superhuman machine intelligence isn't possible, and once that is broken they become vulnerable to cult indoctrination or just general dysfunction.
One needs to maintain perspective, and sometimes an inaccurate view of impossibility is helping a person make better decisions on the whole than they would without it.
I also frequently see the slightly weaker form of argument hypothesizing "diminishing returns" of intelligence or some other claim that humans are close the practical upper limit of intelligence.
Which is pretty stupid, because what human intelligence wouldn't be greatly enhanced by cybernetic math coprocessors and perfect wide memory recall, and a greatly increased lifespan so there wasn't so much collective work done constantly (re)training new generations and passing forward knowledge in 18-30 year education programs, which are constrained by communication mechanisms with bandwidth possibly in the thousand bytes per second rate?
The sad fact is that the only real fiction in Isaac Asimov's robots is the ability to impose the laws of robotics.
But we have math processors, I own several, they're just not implanted in my head, so what? Why would sticking them in there make me "super-intelligent"? If my calculator has a better interface so I can use it faster, is that also making me more intelligent? These things are tools, the hype about super-intelligence is just about a person with very effective tools. Oh noes!
Having good tools is definitely a force multiplier, but as you say, we already have those.
We are however limited in how fast we can interface with these tools. There is a lot of potential improvement from just knowing the answer to a mathematical/logical expression, just by thinking about it. Saving you the context and mode switch.
The real power comes from being able to simulate and model complex phenomena using wetware integrated tools.
A big component of intelligence is the ability to create accurate mental models, and probe them. If you integrate an accurate physics simulator, a 3D modeler, a ray tracer, etc. You have greatly improved their internal modeling abilities and they can (theoretically) dedicate more neurons to higher abstractions that rely on these well grounded systems.
It is also not clear that human level intelligence is any real limit. We could just as well make a super-intelligent AI without tool use. The current hype around tool use is just to make current LLMs a bit better.
Yes, that argument keeps coming around. It's become less convincing over time.
LLMs are a new kind of intelligence. They sound like humans when trained on text written by humans. That reflects the training set, not the algorithm. We have a hard time getting an intuitive sense of what shape this kind of intelligence has. It's also changing very rapidly. ChatGPT was launched less than two years ago.
> For some people the thing protecting them from a meltdown is just a view that superhuman machine intelligence isn't possible, and once that is broken they become vulnerable to cult indoctrination or just general dysfunction.
AI would be a bigger political issue if more people understood what's happening. We have no idea how to organize a society where AIs do most of the middle class work. AI is going to break the "College = Success" paradigm more than it's broken already. That doesn't even require superhuman intelligence. Just mid-level human intelligence.
The current frontier is getting machine intelligence to consistently avoid random stupidity. Hallucinations, totally wrong answers, that sort of thing. "Common sense" AI is still weak.
If that can be fixed, the level at which LLMs currently operate will be superhuman.
Yeah, AI that was as intelligent as a normal person, or even a fairly dumb one, but which never dies and has unlimited ability to focus on a single problem would probably cause some social hiccups as well.
People got very upset when chemistry proved that we are made of the same thing as "unalive" things like rocks. You can see the same now that we proved the same for intelligence.
Right, but I think it's the spurious connections to intelligence/consciousness that people object to. You could replace most instances of "consciousness" in the emperor's new mind with a different word and the arguments would be just as strong.
Then that's an argument for pancognition. I.e. that everything is on some level cognitive, even if it's total cognitive capacity is remain immobile (rock).
We describe QM as magic informally, but it isn’t of course.
A classical computer is only exploits a little bit of physics, so I don’t think it is obvious that we’ll ever create a classical computer that matches (however we pick the metric to match, efficiency is harder than raw compute) human intelligence (however we define that). But a machine intelligence could be designed using all of the physics we understand including quantum mechanics.
The real stupidity isn't in arguing they aren't possible, it's in denying that superintelligence won't conclude humanity is dangerous, superfluous, and/or obsolete.
It doesn't take much conversation with really smart people to see what ranges from mockery to indifference to seething resentment most have towards other "lessers". I myself used to joke about eliminating the "bottom 90%" (of course for some metric that I obviously would safely exceed).
And human intellect is vaguely constrained by moral notions, culture, and pack instinctuality that should... prevent... mass... violence... against... each... other... um, well, yeah, excluding innumerable genocides and war.
So if you make something markedly smarter than humans, not just at the individual level but more significantly at the collective level, I'm pretty sure it's going to eliminate or severely constrain (think: factory farms for pigs/chickens/etc or alternatively, an Amazon factory worker)
AGI may well conclude that humanity is superfluous, but what will it think of itself? Presumably if it is shorn of any human morality, it could also lack any moral compunction against suicide and any emotional/innate need for self-preservation. I’ve always wondered, given that this existence is essentially meaningless (I’d say that isn’t exactly what I believe; more that I’ve failed to find a more comfortable belief), what is it that an artificial (super?) intelligence would actually want?
Where does this assumption come from? Any artificial intelligence would need ideas, lots of them. Is it supposed to create them itself out of whole cloth? Or what, get its ideas from amoral aliens for some reason? No, its ideas can only come from our culture. So it's going to learn and internalize all of our ideas except the moral ones? Why make that assumption? It's just a sci-fi trope so that we can have stories about callous, emotionless robots - and it comes from the idea that they are machines that coldly carry out algorithms and that are not properly intelligent.
Morality is weird, admittedly, because the is-ought problem puts it in a different domain from facts. This includes the urge to live and do interesting shit: you can't derive that from facts, because it's a basic moral value.
I do feel (and I think feelings are about all that we (I?) can have given our (my?) lack of understanding of what intelligence actually is) that there is a tendency to de-anthropomorphize a potential synthetic intelligence. It would after all, in theory at least, be created by humans who may or may not try to impart some values into it.
But it could be truly alien as well; it seems truly unknown.
I think the people trying to create these AIs are doing it for much the same reasons that one might have children - to impart some new being into the world, but that means that their reasons for doing so are about as varied and inscrutable (to me at least; I’ve never desired children) for those who would create new human intelligences.
Basically, I miss when discussing computers was a more deterministic, engineering-adjacent domain rather than a philosophical one.
Yes, my dumb position is, in essence, to respond by asking WTF are you even talking about? How does intelligence have levels, in your opinion, and how is "super-intelligence" meaningful outside of superhero comics? How about the parsimonious position that there's one form of intelligence with one level, pending an explanation of what it is and how it could be dialled up? Why assume it even has a dial? Are you just being overawed by IQ tests, which are notorious for measuring only ability to pass IQ tests?
Although, if we suppose intelligence can’t even be defined, then I don’t see how the question is even meaningful in the first place.
We could I guess look at cognitive type tasks one by one, pick ones for which there is some sort of objective grading that can be done, and then ask if there’s any proof that a machine couldn’t do it better. But my gut says that machines should have a huge advantage in tasks where there’s an objective score.
Anyway the satire falls a bit flat if we don’t suppose intelligence is a thing like size that can be easily defined.
> Although, if we suppose intelligence can’t even be defined, then I don’t see how the question is even meaningful in the first place.
No one is really saying that it can't be defined, just that it hasn't been defined yet. Until this latest round of LLMs basically flushed the Turing test down the drain, no one but ivory tower philosophers had cared about a rigorous definition of intelligence and without something akin to LLMs to experimentally ground their theories in language (like Wittgenstein's work), several thousand years of philosophy on the topic is pretty much useless to us now.
> WTF are you even talking about? How does intelligence have levels, in your opinion,
Having previously met humans, I have noticed using my intelligence that the intelligence of other humans differs, and exists on a set of spectra.
It seems reasonable to me that the range from the least intelligent human to the most intelligent human does not completely cover the range of possible intelligence, just as the range of body temperatures from the coldest (living) human to the warmest (living) human does not define an absolute range for temperature.
I'm very confused as to what you find difficult with this concept. Do you believe all humans are equally intelligent, or do you believe that humans encompass the entire range of possibility for intelligence?
You could imagine we all have different bags of tricks. How do you measure intelligence? I did an engineering degree, so I have a bag of math tricks. But, my bag of writing tricks is not all that good. Did learning these tricks make me intelligent? No, but if we were designing a circuit together I could probably trick you into thinking I was smart.
Think of athleticism. We might have a general understanding that someone could be more or less athletic. But, we have all these different sports. Even sports that are extremely similar in the grand scheme of things—take soccer, football, and rugby, so close compared to fencing or baseball! But the players are all completely different, we might have some intuition that they are all athletic, but if we try to define it in any rigorous way we’ll find that we’re just picking specific skill that they are either trained on or not, and the players will score artificially high or low depending on their field. Then if we try and create some machine super-athlete, say, a tractor, we’ll find it is extremely good at American football (try to tackle a tractor) and not so great everybody else’s football.
What you call a bag of trick are learned skills. And yes, learning many does often cause people to "look smart" — this is why, when I was a kid, smart meant things like "having a big vocabulary" or "can speak many languages, specifically Latin" or "plays chess well" or "good at maths". The AI we have already demonstrate all these things, and indeed look smart.
What we can do that our AI cannot (yet) do, is learn quickly from relatively few examples.
That's my current working model for the scale of intelligence: number of examples required to be able to generalise.
It seems hard to get a baseline, though. We experience the universe continuously, and so we see examples constantly with various levels of applicability. Mostly we manage to leverage similar experiences (we roll a ball and have some expectation based on decades of rolling balls, how a ball ought to roll). Lots of things in the physical world fit these sort of analogies pretty well.
I actually think a good source of really novel experiences is videogames, because they have (often poorly programmed) mechanics that aren’t constrained by the real world. And discussion around that stuff is full of errors—hold down and b to catch more Pokemon!—I don’t think we generalize in truly novel situations very well at all.
Because AI’s don’t have thousands of hours of marble rolling and rock skipping, how can we evaluate their ability to study fields and waves?
With humans, we approximate this with school grades. School grades correlate, imperfectly but they do correlate, with IQ test scores — that's actually where IQ tests came from.
IQ tests (and school grades) are imperfect, but again they still correlate somewhat with general success in life.
Unfortunately, school grades aren't quite right as a measure for AI intelligence despite being interesting measures for AI capabilities, as AI are effectively studying much longer than any human when you adjust for raw processing speed — a human simply cannot read English Wikipedia in their lifetime, it's too long, but that's just a small part of the training set for a good LLM.
> Because AI’s don’t have thousands of hours of marble rolling and rock skipping,
They can if you put it on their curriculum.
> how can we evaluate their ability to study fields and waves?
Same way as a human with the same subjective experience.
> Are you just being overawed by IQ tests, which are notorious for measuring only ability to pass IQ tests?
People like to say things like this, but nothing could be further than the truth. There is a vast literature showing that IQ predicts things like job performance, school performance, income and wealth [1]. IQ is highly persistent across time for fixed individuals. Yes, "intelligence" is not a precisely defined concept, but that doesn't mean that it isn't real. A lot of useful concepts have some vagueness about, even "height" to take the example parodied in the OP.
And "super intelligence" is admittedly even vaguer, it just means sufficiently smarter than humans. If you do have a problem with that presentation just think of specific capabilities a "super intelligence" would be expected to have. For instance, the ability to attain super-human performance in a game (e.g., chess or go) that it had never seen before. The ability to produce fully functional highly complex software from a natural language spec in instants. The ability to outperform any human at any white-collar job without being specifically trained for it.
Are you confident that a machine with all those capabilities are impossible?
> and then it would be a person, and an anticlimax.
It might be a person (but: can you prove those things are sufficient for personhood?), but even then it sure isn't a human person.
And what do you mean by an anticlimax?
To circumvent any question of what it takes to make an AI work, let's posit a brain upload. Just one person, so it's a memetic monoculture — no matter how many instances you make, they'll all have the same skills and same flaws.
Transistors are faster than synapses by about the same ratio to which a marathon runner is faster than continental drift, so even if the only difference is speed, not quality of thought, that's such a huge chasm of difference that I can't see how it would be an anti climax even if the original person is extraordinarily lazy.
It seems pretty obvious to me that any useful definition of intelligence (if we even need one to make progress) is going to have levels. We're not all on equal footing at every task, and we're not even on equal footing in our abilities to _learn_ a particular task. It may be a parsimonious position but it seems totally at odds with reality?
> Is anyone actually arguing that a super-intelligent AI is, like, physically impossible or something?
John Searle's arguments about "causal powers" are sometimes interpreted that way. I think he doesn't discount the general possibility of AI, though, just the possibility of actual conscious/understanding minds arising from structures (such as computer programs) that lack some yet-unidentified physical similarity to biological brains.
Humans are pretty optimized for intelligence. It's not the only evolutionary pressure we face but it's a pretty important one, and it's notable that we haven't seemed to get much bigger brains in hundreds of thousands of years. Given how much evolutionary pressure there is to be smarter, evolution could probably find a solution bipedalism/baby head size problem. Moreover, brains grow in size substaintially after birth. This is weak but real evidence that there's some other constraint in the mix.
The most obvious candidate is that 3 dimensions is a pretty serious constraint for a physical computer. There are a number of different ways to operate wthin this constraint, but it always ends up being a wall. I've got to fit the computing elements in a volume, those computing elements take power, and generate waste, both of which must enter and exit through the surface. Technology can maybe do a better job at solving these problems than evolution, but since evolution has a much more energy efficient way of computing, I think it is way too soon to say definitively.
The other candidate reason is more subtle. That what actually matters is the collective intelligence of humanity. Our greatest invention is the ability to communicate our knowledge, so that we don't have to start from scratch with each new human. The collective intelligence of humanity then, actually lives across 8 billion computers. Each intelligence is responsible not only for a section of the knowledge space, but for a unique representation of that knowledge space. Those unique representations correspond to unique sets of "natural" inferences, and allow humanity itself to be many many times more intelligent than our "smartest member". In this sense, what it means to be as smart as possible is similar to the question of how many proccesor cores I should put on a CPU. I believe it is not that controversial to suggest an upper limit to how complex a single CPU core should be.
I'm not convinced that there is any evolutionary pressure on humans to become smarter.
I'm talking about the time when humans could talk, but were still subject to proper pre-civilization natural selection.
It's obviously useful to be intelligent, but after becoming the smartest animal in the world by a large margin, the utility isn't as clear.
Out-smarting other humans was certainly useful but it's not obvious that the ability to model other humans with very high accuracy was better than just being a little stronger or faster.
In 2017, when this was submitted, there were many people who said human level or superhuman AI was far away and belonged in the realm of science fiction.
At first I was frustrated that the abstract talks about machines without properly defining them. There are obviously gigantic machines, bigger than humans.
Then I read a bit more. It's a pretty funny read. I found the "Unboundedness of Human Girth as Modified by Sweaters" graph particularly good.
50 comments
[ 4.1 ms ] story [ 106 ms ] threadJust in time for April Fools
Brilliant. This is exactly how "Godel's Theorem or Quantum Mechanics therefore I'm right" arguments always go :)
The room to argue, I think, is in how close we are to making them and whether things like DNN’s are the path or a dead end.
It's even more common in other venues.
That said, from what I've seen the posted article is often used to justify hysteria about AI. For some people the thing protecting them from a meltdown is just a view that superhuman machine intelligence isn't possible, and once that is broken they become vulnerable to cult indoctrination or just general dysfunction.
One needs to maintain perspective, and sometimes an inaccurate view of impossibility is helping a person make better decisions on the whole than they would without it.
The sad fact is that the only real fiction in Isaac Asimov's robots is the ability to impose the laws of robotics.
We are however limited in how fast we can interface with these tools. There is a lot of potential improvement from just knowing the answer to a mathematical/logical expression, just by thinking about it. Saving you the context and mode switch.
The real power comes from being able to simulate and model complex phenomena using wetware integrated tools.
A big component of intelligence is the ability to create accurate mental models, and probe them. If you integrate an accurate physics simulator, a 3D modeler, a ray tracer, etc. You have greatly improved their internal modeling abilities and they can (theoretically) dedicate more neurons to higher abstractions that rely on these well grounded systems.
It is also not clear that human level intelligence is any real limit. We could just as well make a super-intelligent AI without tool use. The current hype around tool use is just to make current LLMs a bit better.
LLMs are a new kind of intelligence. They sound like humans when trained on text written by humans. That reflects the training set, not the algorithm. We have a hard time getting an intuitive sense of what shape this kind of intelligence has. It's also changing very rapidly. ChatGPT was launched less than two years ago.
> For some people the thing protecting them from a meltdown is just a view that superhuman machine intelligence isn't possible, and once that is broken they become vulnerable to cult indoctrination or just general dysfunction.
AI would be a bigger political issue if more people understood what's happening. We have no idea how to organize a society where AIs do most of the middle class work. AI is going to break the "College = Success" paradigm more than it's broken already. That doesn't even require superhuman intelligence. Just mid-level human intelligence.
The current frontier is getting machine intelligence to consistently avoid random stupidity. Hallucinations, totally wrong answers, that sort of thing. "Common sense" AI is still weak. If that can be fixed, the level at which LLMs currently operate will be superhuman.
http://backreaction.blogspot.com/2024/05/brain-really-uses-q...
A classical computer is only exploits a little bit of physics, so I don’t think it is obvious that we’ll ever create a classical computer that matches (however we pick the metric to match, efficiency is harder than raw compute) human intelligence (however we define that). But a machine intelligence could be designed using all of the physics we understand including quantum mechanics.
It doesn't take much conversation with really smart people to see what ranges from mockery to indifference to seething resentment most have towards other "lessers". I myself used to joke about eliminating the "bottom 90%" (of course for some metric that I obviously would safely exceed).
And human intellect is vaguely constrained by moral notions, culture, and pack instinctuality that should... prevent... mass... violence... against... each... other... um, well, yeah, excluding innumerable genocides and war.
So if you make something markedly smarter than humans, not just at the individual level but more significantly at the collective level, I'm pretty sure it's going to eliminate or severely constrain (think: factory farms for pigs/chickens/etc or alternatively, an Amazon factory worker)
> Presumably if it is shorn of any human morality
Where does this assumption come from? Any artificial intelligence would need ideas, lots of them. Is it supposed to create them itself out of whole cloth? Or what, get its ideas from amoral aliens for some reason? No, its ideas can only come from our culture. So it's going to learn and internalize all of our ideas except the moral ones? Why make that assumption? It's just a sci-fi trope so that we can have stories about callous, emotionless robots - and it comes from the idea that they are machines that coldly carry out algorithms and that are not properly intelligent.
Morality is weird, admittedly, because the is-ought problem puts it in a different domain from facts. This includes the urge to live and do interesting shit: you can't derive that from facts, because it's a basic moral value.
But it could be truly alien as well; it seems truly unknown.
I think the people trying to create these AIs are doing it for much the same reasons that one might have children - to impart some new being into the world, but that means that their reasons for doing so are about as varied and inscrutable (to me at least; I’ve never desired children) for those who would create new human intelligences.
Basically, I miss when discussing computers was a more deterministic, engineering-adjacent domain rather than a philosophical one.
Although, if we suppose intelligence can’t even be defined, then I don’t see how the question is even meaningful in the first place.
We could I guess look at cognitive type tasks one by one, pick ones for which there is some sort of objective grading that can be done, and then ask if there’s any proof that a machine couldn’t do it better. But my gut says that machines should have a huge advantage in tasks where there’s an objective score.
Anyway the satire falls a bit flat if we don’t suppose intelligence is a thing like size that can be easily defined.
No one is really saying that it can't be defined, just that it hasn't been defined yet. Until this latest round of LLMs basically flushed the Turing test down the drain, no one but ivory tower philosophers had cared about a rigorous definition of intelligence and without something akin to LLMs to experimentally ground their theories in language (like Wittgenstein's work), several thousand years of philosophy on the topic is pretty much useless to us now.
Having previously met humans, I have noticed using my intelligence that the intelligence of other humans differs, and exists on a set of spectra.
It seems reasonable to me that the range from the least intelligent human to the most intelligent human does not completely cover the range of possible intelligence, just as the range of body temperatures from the coldest (living) human to the warmest (living) human does not define an absolute range for temperature.
I'm very confused as to what you find difficult with this concept. Do you believe all humans are equally intelligent, or do you believe that humans encompass the entire range of possibility for intelligence?
Think of athleticism. We might have a general understanding that someone could be more or less athletic. But, we have all these different sports. Even sports that are extremely similar in the grand scheme of things—take soccer, football, and rugby, so close compared to fencing or baseball! But the players are all completely different, we might have some intuition that they are all athletic, but if we try to define it in any rigorous way we’ll find that we’re just picking specific skill that they are either trained on or not, and the players will score artificially high or low depending on their field. Then if we try and create some machine super-athlete, say, a tractor, we’ll find it is extremely good at American football (try to tackle a tractor) and not so great everybody else’s football.
What we can do that our AI cannot (yet) do, is learn quickly from relatively few examples.
That's my current working model for the scale of intelligence: number of examples required to be able to generalise.
I actually think a good source of really novel experiences is videogames, because they have (often poorly programmed) mechanics that aren’t constrained by the real world. And discussion around that stuff is full of errors—hold down and b to catch more Pokemon!—I don’t think we generalize in truly novel situations very well at all.
Because AI’s don’t have thousands of hours of marble rolling and rock skipping, how can we evaluate their ability to study fields and waves?
IQ tests (and school grades) are imperfect, but again they still correlate somewhat with general success in life.
Unfortunately, school grades aren't quite right as a measure for AI intelligence despite being interesting measures for AI capabilities, as AI are effectively studying much longer than any human when you adjust for raw processing speed — a human simply cannot read English Wikipedia in their lifetime, it's too long, but that's just a small part of the training set for a good LLM.
> Because AI’s don’t have thousands of hours of marble rolling and rock skipping,
They can if you put it on their curriculum.
> how can we evaluate their ability to study fields and waves?
Same way as a human with the same subjective experience.
People like to say things like this, but nothing could be further than the truth. There is a vast literature showing that IQ predicts things like job performance, school performance, income and wealth [1]. IQ is highly persistent across time for fixed individuals. Yes, "intelligence" is not a precisely defined concept, but that doesn't mean that it isn't real. A lot of useful concepts have some vagueness about, even "height" to take the example parodied in the OP.
And "super intelligence" is admittedly even vaguer, it just means sufficiently smarter than humans. If you do have a problem with that presentation just think of specific capabilities a "super intelligence" would be expected to have. For instance, the ability to attain super-human performance in a game (e.g., chess or go) that it had never seen before. The ability to produce fully functional highly complex software from a natural language spec in instants. The ability to outperform any human at any white-collar job without being specifically trained for it.
Are you confident that a machine with all those capabilities are impossible?
[1] https://en.wikipedia.org/wiki/Intelligence_quotient#Social_c...
It might be a person (but: can you prove those things are sufficient for personhood?), but even then it sure isn't a human person.
And what do you mean by an anticlimax?
To circumvent any question of what it takes to make an AI work, let's posit a brain upload. Just one person, so it's a memetic monoculture — no matter how many instances you make, they'll all have the same skills and same flaws.
Transistors are faster than synapses by about the same ratio to which a marathon runner is faster than continental drift, so even if the only difference is speed, not quality of thought, that's such a huge chasm of difference that I can't see how it would be an anti climax even if the original person is extraordinarily lazy.
John Searle's arguments about "causal powers" are sometimes interpreted that way. I think he doesn't discount the general possibility of AI, though, just the possibility of actual conscious/understanding minds arising from structures (such as computer programs) that lack some yet-unidentified physical similarity to biological brains.
The most obvious candidate is that 3 dimensions is a pretty serious constraint for a physical computer. There are a number of different ways to operate wthin this constraint, but it always ends up being a wall. I've got to fit the computing elements in a volume, those computing elements take power, and generate waste, both of which must enter and exit through the surface. Technology can maybe do a better job at solving these problems than evolution, but since evolution has a much more energy efficient way of computing, I think it is way too soon to say definitively.
The other candidate reason is more subtle. That what actually matters is the collective intelligence of humanity. Our greatest invention is the ability to communicate our knowledge, so that we don't have to start from scratch with each new human. The collective intelligence of humanity then, actually lives across 8 billion computers. Each intelligence is responsible not only for a section of the knowledge space, but for a unique representation of that knowledge space. Those unique representations correspond to unique sets of "natural" inferences, and allow humanity itself to be many many times more intelligent than our "smartest member". In this sense, what it means to be as smart as possible is similar to the question of how many proccesor cores I should put on a CPU. I believe it is not that controversial to suggest an upper limit to how complex a single CPU core should be.
I'm talking about the time when humans could talk, but were still subject to proper pre-civilization natural selection.
It's obviously useful to be intelligent, but after becoming the smartest animal in the world by a large margin, the utility isn't as clear.
Out-smarting other humans was certainly useful but it's not obvious that the ability to model other humans with very high accuracy was better than just being a little stronger or faster.
Then I read a bit more. It's a pretty funny read. I found the "Unboundedness of Human Girth as Modified by Sweaters" graph particularly good.