My pet theory is that anything we regard as truly intelligent has to have a sense of self, an identity if you will. In addition, it has to be motivated by something. All living things, and all the intelligent living things, have a survival and reproduction instinct on some level which is an animating force driving higher level thinking and actions. Basically, any AGI has to have something approaching a ‘soul’ to be ‘intelligent.’ And if that emerges, combined with exponential iterative evolution that isn’t limited by biology ... well all the sci-fi tropes seem plausible.
I think semi-retiring. He's going to be "consulting [on Oculus]" while:
> I am going to be going about it “Victorian Gentleman Scientist” style, pursuing my inquiries from home, and drafting my son into the work.
Which to me reads like part time work on Oculus, part time work on this AGI project. If it is with Facebook it isn't at all clear from the post (plus I'd assume it would be accompanied by marketing copy in that situation).
>Starting this week, I’m moving to a "Consulting CTO” position with Oculus.
I will still have a voice in the development work, but it will only be consuming a modest slice of my time.
As for what I am going to be doing with the rest of my time:
[...]
For the time being at least, I am going to be going about it “Victorian Gentleman Scientist” style, pursuing my inquiries from home, and drafting my son into the work.
Having listened to his appearance on the Joe Rogan Experience, I can't help but hear this message in my head with his distinct voice. But also, I think his brutal honesty and relentless work ethic will suit this problem domain.
I hope he'll go the hybrid symbolic and neural network way (causal and statistical), instead of just statistical.
AGI needs a type system...
I hope I'll achieve AGI before him but it's nice to know there's some real competition!
(because, reader, there are almost 0 researchers seriously trying to achieve AGI in a not totally bullshit way.
Only opencog and Cyc comes to mind).
Those of us who've been on the net long enough, and have at least dabbled in AI/ML circles, know of him.
He claims to have invented a program that is a mind, originally written in Forth, translated by others to many other languages, etc. He has published the code of this program in a form of "open source", so you can easily find it if you dig enough.
He's widely considered to be a crank. That said, the line between genius and madness can be mighty thin, and what side he lands on is anyone's guess, but most put him well over the line into madness, for whatever its worth.
My own opinion?
Well - looking at his work from purely the modern understanding and research into ML/AI - that is, deep learning and such - his work would be considered pointless, probably worse than Eliza as to its contributions to the field.
But as someone who has read a lot of other works (for for and against) the idea of AGI, artificial consciousness, theory of mind, etc - his work at a certain level has echos with some of that work. Still probably a dead end, but at the same time, there's some interesting concepts within his code and theories (he's self-published a book on it, too - you can find it on Amazon - he also has it for free on his github and it can be found elsewhere).
I guess I still put him in crank territory, but not in the abusive crank arena, more in the "doing his own thing, but being a bit evangelical about it" - relatively harmless.
His work is not as amazing as TempleOS, imho, but there's a similar mind behind it (though comparing it with that operating system is maybe an unfair, possibly orthogonal, comparison).
I won't say or reveal more (but I've written enough for you to figure it out) - he tends to monitor tons of forums and if he thinks he's being "summoned", he'll spam the forum with his writings and theories. It got him "perma-banned" from more than one newsgroup back in the day...
You might just begin by learning the list of NLP tasks and how good are the state of the art at it.
The cognitive architecture that needs to be created to achieve AGI will one way or another be a composition of said tasks, which are the primitives.
I'm not a big player on the field.
I'm specialized in semantic parsing and argument checking.
I'm the first to my knowledge to have made a syllogism (and more) checker for English.
Also, I have allowed researchers to beat the state of the art on constituency and dependency parsing (but simply by sharing knowledge of the state of the art to other researchers).
I do this on my free time so I'm not productive, but I have designed an intermediary language (IR) for natural languages that seems very promising.
Are you really that sure that the approaches to increasing the generality of AI being taken by LeCun (self-supervised model learning), Hinton (capsule networks) and Bengio (state representation learning) are all "total bullshit"?
From what I've actually seen that is completely inaccurate and unfair. The only thing that he did to "deserve that" that I know of is to start seriously pursuing and talking about AGI before it was cool again.
For example, OpenCog is an implementation of the classic cognitive architecture and its about as traditional and far from "total bs" as you can get in AGI.
I have never heard anything to back up the insults against Goertzel.
From my reading, Hinton capsule networks seems far from being enough, it could at best be an incremental improvement.
And is unrelated to English semantic parsing, it seems specialized for computer vision.
This is a very artfully written statement. It avoids the most more devastating headline of “John Carmack Stepping Down”. (Yes, going part-time isn’t the same as leaving entirely, but still).
More like "John Carmack has well-balanced life priorities and has decided to do a really cool moonshot project with his son instead of making even more money"
Crap! When I thought VR was hitting its stride with the oculus quest.
For anybody who hasn't played an untethered VR experience, I highly recommend it. It makes a world of difference with games like Echo combat and Beatsaber. Tons of fun. It reminds me of the first time I played wii bowling.
The real headline is that VR is finally confirmed to have hit a major roadblock. If John Carmack gets frustrated with its progress, there's something up.
I've long thought that the issue with VR is a conceptual one, not a technical one and maybe that frustration comes from there. "Running forward" is an unsolved problem in room scale VR. For a seated experience, you're basically back to a neat display gimmick + accurate hand tracking.
Any real solutions need, on the one side, real-world physical constructions (think running threadmills) that soon hit holodeck-level limitations and, on the other, software that actually benefits from the real technology VR brings to interactive media: super accurate hand- and head-tracking. The first gets impractical/impossible soon, the second limits development to a few niche genres: Shooting ranges, cockpit sims, dance/party games and some vague "experiences" where the actual tech is pretty much ignored and you just say "but it feels so immersive!" (honestly, it does work for horror games!). It's basically motion controls 2.0.
The only place I could see the technology shine is, oddly enough, AR. It has way less mainstream hype to it but it makes much more sense because you actually benefit from the tracking of your real-world movement: You're still a part of it! The holo-lens demos that pop up on youtube might seem clumsy, but I can totally see a use case for replacing physical monitors with arbitrarily sized and positioned displays you can virtually move in any office space. There's rumors of Apple working with Valve on AR tech. If there's any technology that could follow the smart phone, AR is my bet. I'm honestly surprised Carmack didn't move in that direction rather than deciding to become a general AI guru.
> The real headline is that VR is finally confirmed to have hit a major roadblock. If John Carmack gets frustrated with its progress, there's something up.
This was the (loud) subtext for me, too.
Carmack doesn't strike me as the type of person to walk away from a problem lightly. Given how many problems remain unsolved when it comes to VR, I wonder if he's just admitted that it will never be the endgame he wanted it to be.
Cash-wish, he's obviously sitting pretty. Better off spending your remaining years working on something you can make a meaningful contribution towards.
I don't understand how he could contribute to the field of AGI research from home, by himself, and maybe with his son. It's the kind of problem that requires incredible amounts of data, hardware, and theory to make any progress.
Wouldn't it make more sense for him to join a cutting-edge team, like DeepMind or OpenAI?
Everything I've ever read about Carmack suggests he'll do his best on his own at home. Much of this work can be done on reasonable hardware, and he's always been really good at getting a lot out of reasonable hardware. Further, if he needs enormous compute resources, he can get it at any of several cloud providers.
> if he needs enormous compute resources, he can get it at any of several cloud providers
This is exactly the experience of most teams I've spoken with, be they students or businesses, for all the pre-production phase. You simply can't and shouldn't spend on costly AI infrastructure before you've nailed your solution; in fact any kind of infra not just AI.
What you do is rent some cloud to power quickly through your tests — better have 10x worth of big Nvidia GPUs over 2 weeks than buy 1 or 2 max yourself and wait 5-10x more time — not even factoring that setting up clusters of GPU and running such workflows consistently over days, weeks requires pretty deep sysadmin/hardware knowledge and experience; it took me two years to really master that non-problem part on my home server (but now it's a skill I have so that was worth it, but certainly set my research and learning back by as much time).
Besides, there's a time when the familiarity, safety and general comfort of home simply can't be beat. Notwithstanding pool tables and free soda, lol.
Might be a little burnt out from the "cutting-edge team" at Oculus who painted him into a corner with technology that could never work, as it made every user sick.
It might make sense for him to join a team like DeepMind, but we could guess that the "working from home by himself" bit was a lifestyle change he wouldn't compromise on.
I think many people expect that a lot of the missing "special sauce" for AGI (if anyone can figure it out at all) is going to be something for which massive GPU power isn't a key factor.
Which will be the cognitive part. The machine learning is more like perception. But perception needs to be tied into an understanding of the world where inferences can be made and one can adjust quickly to a changing environment, while learning new domains or even creating new combinations. This also includes the social-emotional world of humans and language (and not just translation), of course.
Maybe there is no secret. Just like image recognition is just a bunch of well connected matrices running a dumb algorithm, but at a great speed by GPUs, intelligence is just 100 billions dumb nano-computers with the logic of a fairly simple finite state automata, but with 10 thousand network connections per node. How does nematoda transfer intelligence to its copies? By encoding the FSA properties in the DNA. If this is the case, we'll see the next chapter of AI once a typical smartphone runs a million dumb programmable nanocomputers with a very sense network topology: people will just run the same dumb algorithms on this devices and discover that it exhibits the basic properties of nematoda-level AI. And thus AI would be a dumb engineering problem.
My view is that you want many people working independent from each other towards the same goal, and that everyone working in one group could hinder creativity/lead to groupthink
He's openly talked about his work ethic in a bunch of places. He's the type of guy who after a life time of coding calculated he's 100% efficient up until 13 hour work days and then he drops off[0]. Although he did mention working those long hours is often best working on multiple things instead of 1 topic but maybe with AGI there's a bunch of different avenues to explore.
Uuh, AI research has nothing to do with coding all nighters. This is a common misconception among software engineers. It is more a science, and less an engineering problem. It is more about running experiments than it is writing fancy algorithms.
You are bound by the amount of data and computational resources you have at your disposal. Neither are tied to man hours. You can stay up all night for days waiting for your model to train, and it will do you no good.
Carmack built rockets and id bought $100Ks of NeXT machines to make Doom so I wouldn't put it past him to have incredible amounts of hardware... even at home. Considering his position at Facebook and that he is industry famous he probably has access to data and cloud resources that a researcher outside of OpenAI, Nvidia, Google, etc. normally wouldn't have access to. He could also raise money relatively easily to pursue more intense research.
What would be awesome is if he just said one day "I need 100 million dollars for my AGI project to buy hardware, anyone who wants to share in a 20% cut of the business just send funds to bitcoin address ### or ethereum address ###". He would be fully funded within an hour, probably.
Unfortunately that could never happen because of the SEC.
> It's the kind of problem that requires incredible amounts of data, hardware, and theory to make any progress.
I wouldn't be surprised if the opposite was true, at least with the theory part. AI didn't really go anywhere for decades, because people focused too much on theory.
Otherwise, there's a lot of data and hardware at your disposal, even from the comfort of your home.
> Wouldn't it make more sense for him to join a cutting-edge team, like DeepMind or OpenAI?
You mean they guys that are training with videogames that people like John developed?
Deepmind and openai are probably not on a reasonable track to AGI. IMO if we ever make an AGI it wont actually be especially good at things. An AGI, like humans, would probably be pretty bad at math naturally. You could get one to be great at math, but first getting great at math and backing into general intelligence is probably impossible.
> It's the kind of problem that requires incredible amounts of data, hardware, and theory to make any progress.
So I point out a recent example of progress (of which the sequence of fundamental insights is nearly always incremental), where the theoreetical insight was a theoretical derivation, which could be and probably was derived on paper / blackboard, as a direct counterexample: it does not require "incredible amounts of [...] theory".
Why would you compare this with performing manual backpropagation?
>Everything is easy in hindsight
Every breakthrough is non-trivial, else it would not have been a breakthrough, and yet the breakthrough itself can be a relatively simple calculation...
The concept of "AGI" is undefined and virtually worthless to me. There is only non-trivial insights, i.e. theorem and proof.
i say this with respect and humility, but i am very surprised at the naivete with which John addressed the subject of AGI. he is so casual about it -- not only the idea of working on it but also the idea of it existing at all. he seems oblivious to the gravity of that discovery. it is not just "very valuable," it will be earth shattering and probably wipe out humanity. and its his side-project. and his son will help out.
John is the perfect representation of what is wrong with peoples attitude toward AGI. aloof and naive.
What is your source for saying AGI will probably wipe out humanity? How could we ever even attempt to guess at the motivations of something we can barely comprehend and doesn't even exist yet?
The main concern is not that it's like Skynet and wishes us harm, but that it does things harmful to us because the means to accomplish the AI's goals are at odds with human values, which was unanticipated by the human creators, since the AGI is coming up with its own solutions. And the AGI doesn't have the same values as humans, so it doesn't care if its solutions are harmful.
I guess the risk seems more real if it's a question of "does it share human values". Which is possibly not a one in a million chance it doesn't - it could be a 50/50 chance or worse.
I'll say this: I'd prefer if the brightest minds approached the matter from the "AI safety" angle (a sub field concerned with building not just AI but "safe" AI, ie that we can control or understand in a practical manner).
Because that's really where the line of human history will be drawn if AGI and above becomes real. AI safety, how advanced we are in it, will directly map to civilization's progress or endangerment as a result of AI.
Edit: this is already true with regards to "psychological safety" from undue influence or outright manipulation with motive (usually financial) by current "ANI" algorithms (newsfeeds, "recommendations", ads, etc). It's a real topic that reduces to human psychological freedom, freewill. It's a BIG topic.
It would be easier to take the AI doomsayers' seriously if we were remotely close to AGI. For now it's treated the same way as some guy in a cape in Central Park trying to summon Satan. No one cares because everyone knows it's basically impossible.
I'll level with you, if all I had heard was soundbites, I'd be skeptical myself; it's a bit like Neil DeGrasse Tyson telling you he's going to unify gravity or something. This guy can go and adlib a 2 hour talk about implementing subsurface scattering, and build it from the ground up in a commercially viable way.
This dismal, dimwitted "advanced" filter and sort industry has recently started training all their employees in and vomiting all over every consumer with is nothing, worthless, and at best lunacy.
All it takes is patience, know-how, and insight. For that, Carmack fits the bill.
You are saying "probably wipe out humanity" is a bad thing ?
I know of several million species that would strongly disagree. Especially if AGIv1 decides to tune the genetics of say most mammals to append 'sapient' to the end of their species name.
Perhaps more constructively consider that AGI is simply the next iteration of 'humanity', yea sure the old versions are redundant anarchisms and apart from some living reserve specimens functionally extinct, but nobody cares as you can sim one up at almost no cost.
AGI is not the next iteration of humanity because it will not resemble humanity in any way besides being sentient in some capacity. you will feel quite silly if you get to see it in your lifetime.
You know of 0 species that would strongly disagree, because those several million species don't have the capacity to agree or disagree on whether "wiping out humanity is a bad thing". And you don't speak for them, no matter how much caring about the planet you think you're doing.
I absolutely speak for them, who else will ? Clearly not people like you. I have the capacity to speak for them and so I do. And I will be the only judge about how much "caring" I do or don't do.
You guys really get high on your own supply don't you? It's a toss-up these days whether something from computer science or a programmer will either enhance or erode the human experience and quality of life.
It's more: offer substantiative evidence of your counter-culture claims that vilify most of your reading audience, or take your baseless negativity elsewhere
One-off snide attacks in a comment section without any evidence or argument to back it up offers nothing of value.
If you have a strong stance about the evils of software development then write something of substance and post it. I am sure it will get plenty of discussion if there's any merit to it.
> One-off snide attacks in a comment section without any evidence or argument to back it up offers nothing of value.
Can you back that up? Here's the comment:
> You guys really get high on your own supply don't you? It's a toss-up these days whether something from computer science or a programmer will either enhance or erode the human experience and quality of life.
"getting high on your own supply" isn't such a giant insult that you can simply "make the dozen full" and claim whatever you want about the comment.
> I am sure it will get plenty of discussion if there's any merit to it.
Then you're sure of something that is demonstrably false, and one-off low effort sophistry like this offers nothing of value. There is no hard connection between the merit of something and HN's ability to discuss it. Take this story sticking to the top for probably over 24 hours, basically an announement to spend more time with his son, and that's it, while this got sunk off the front page yesterday: https://news.ycombinator.com/item?id=21527622
The forum of a VC firm doesn't seem antithetical to that at all.
No one here is claiming that software development doesn't make fat stacks and viable businesses. It just happens to do it by perverting humanity. Moving us further away from our ideal environment.
Even a decade ago, I'd be all onboard seeing computer science as "advancing humanity." Lately not so much so. Maybe the comparison to phsyics is appropriate. We have reached the "nuclear weapons" age of computer science where we need programmers to make ethical stands, lest we severely damage the human condition.
That argument is like saying if we had better theorists you wouldn't need to build particle accelerators.
Nature has brute facts that can only be discovered through observation. I would be surprised to see evidence that an intelligence, no matter how smart, could reason to the fundamental properties of neutrinos without massive physical real world experiments and piles of observational data.
If it could, that would mean rationalism as a philosophy would come back into play, whereas empiricism has been dominant during the scientific revolution.
why would being born in the current century preclude one from making fundamental contributions?
You are making a common mistake of assuming that just because someone is good at something like programing computers the same skill would translate identically to a completely different domain.
If anything he is lucky to have been born in an era where his skill of programming computers could be put to use - otherwise his talents may have gone to waste, he may have ended up toiling fields his talent untapped and undiscovered, like that of millions before him.
High intelligence in one technical domain translates well enough to competence in other technical domains. Carmack isn't just a programmer, he is a highly creative technical problem solver. However, AGI does seem orders of magnitude more complex than developing 3D game engines.
John in particular has shown a propensity to translate his skills into disjoint domains pretty well. I don't think he's good at what he does because he's good at computers. I think it's more that he's really really good at understanding problems and designing reasonable paths to solutions.
And then he has the ability to power through learning what he needs to in order to build toward those solutions lightning fast.
He's "good at programming" like Galileo was "good with telescopes." Computers, telescopes, hammers, they're all just tools, and Carmack has proven himself as far more than just a handyman.
He's the kind of person where, if you show him what you're working on and he doesn't understand it, you probably need to go back to the drawing board.
Your argument is a bit like: here is this amazing long-distance runner who trains for years at a time, had he put all that effort into painting he would be a new Picasso.
For what is worth computers are unique tools unlike any other tool that mankind has invented before - thus it is much harder to tell what other jobs would a good programmer excel at.
Cult of personality much? Don't get me wrong, Carmack is one of a kind. But seriously, "if Carmack doesn't understand your idea => your idea is hopeless" -- this can not be healthy.
EDIT: I'll elaborate a bit. In my experience in both industry and academia I've witnessed numerous occasions when brilliant people would get things wrong, ignore a brilliant idea, follow a hopeless research direction, etc. etc. Authority matters, but _nobody_ is flawless.
> John Carmack seems like a guy that would've made fundamental contributions to science, had he been born a century before.
I see Carmack as a very(as in uniquely) talented Engineer. Usually, engineers are not the type who do very well in pure research topics. And AGI is certainly a pure research topic, since we don't have a clear leading us there. So while it's great to see he is interested in it, now sure if we should have any kind of expectation there.
Progress in AI is due to data and computational power advances. I wonder what kind of advances are needed for AGI.
1. Biological brains are non-differentiable spiking networks much more complicated than backpropagated ANNs.
2. Ion channels may or may not be affected by quantum effects.
3. The search space is huge (but organisms aren't optimal and natural selection is probably local search)
4. If it took ~3.8b years to get from cells to humans, how do we fast-forward:
* brain mapping (replicating the biological "architecture")
* gene editing on animal models to build tissues and/or brains that can be interfaced (and if such interface could exist how do we prevent someone from trying to use human slaves as computers? Using which tissues for computation is torture?)
* simulation with computational models outside of ECT (quantum computers or some new physics phenomenon)
Note: those 3.8b years are from a cell to human. We haven't built anything remotely similar to a cell. And I'm not claiming that an AGI system will need cells or spiking nets, most likely a lot of those are redundant. But the entropy and complexity of biological systems is huge and even rodents can outperform state of the art models at general tasks.
IMHO, the quickest path to AGI would be to focus on climate change and making academia more appealing.
You forgot to mention, crucially, that neurons in close proximity affect each other, which is just one of the things that makes modeling of more than a few neurons in time domain a complete non-starter. It all results in enormous systems of PDEs which we don't know how to solve yet at all. You could say that we do not have the right mathematical apparatus to model any such thing.
Pure physical modeling are likely a bad representation for the phenomena resulting in intelligence, especially granted we've simulated it with much simpler discrete structures. PDEs may even be disastrously bad, like trying to describe a line in space with a table of points, instead of the degrees of freedom.
I would imagine that a PDE may cover diffuse behaviors governing say, how learning happens mechanically, but it is almost certainly a language/representational barrier, the relationship between the structure of the animal mind, learning, and seemingly simple phenomena, like afterimages.
The molecules are arbitrary and the timescale doesn't matter.
I don't follow that. What would prevent (perhaps quite slow) simulation of a larger system of such neurons? E.g. N-body problems are analytically beyond us, but can be simulated to arbitrary precision with certain trade-offs.
Time domain solutions do not exist for more than a dozen neurons. At least they did not when I took a computational neuroscience MOOC a couple of years ago. State of the art at the time was the nervous system of an earthworm. That is, if you consider what you actually need to do to simulate how potentials will change in the brain over time give a certain starting state and stimuli, the math gets so complicated (and awkward) so quickly that it's not really tractable with the mathematical (or simulation) apparatus we currently have to go beyond such trivial systems.
From what I understand, quantum effects being essential to the process is a fringe belief. Penrose is probably the most famous 'serious person' (sorry Deepak Chopra) to espouse the idea, but I'm inclined to believe that might be a Linus Pauling/Vitamin C sort of scenario. Penrose started from the perspective of believing there must be quantum effects, then began fishing for physical evidence of it.
I was taught that the quantum theory of memory and cognition generally falls under Eric Schwartz's "neuro-bagging" fallacy [0]. That is:
>You assert that an area of physics or mathematics familiar to few neuroscientists solves a fundamental problem in their field. Example: "The cerebellum is a tensor of rank 10^12; sensory and motor activity is contravariant and covariant vectors".
So yeah, I feel that it's pretty fringe (as you suggested).
I think you're recalling the end of this comic[1], which was on the front page of HN a couple weeks ago. So the quote is probably attributable to either Scott Aaronson or Zach Weinersmith.
"The Secret of Scent" by Luca Turin [0] if I remember correctly goes into research that indicates that there may be quantum effects that explain how shape/chirality of molecules affect smell.
[0] https://www.amazon.com/Secret-Scent-Adventures-Perfume-Scien...
So it is plausible that nature may have evolved to be affected by quantum effects.
I'm sure neural nets will herald AI right after the mechanical gears and pneumatic pistons that were envisioned as the secret sauce during the turn of the last century.
The key, of course, is redefining life and intelligence as whatever the current state-of-the-art accomplishes. (Cue explanations that the brain is just a giant pattern matcher.) It makes drawing parallels and prophesying advancements so much easier. Of all our sciences, that's perhaps the one thing we've perfected--the science of equivocation. And we perfected it long ago; perhaps even millennia ago.
Gene expression is often tied to the environment the organism is in. Mere possession a gene isn't enough to benefit from it. Some expressions don't take effect immediately, but rather activate in subsequent generations.
Epigenetics is a whole equally large layer on top of this system. A single-focus approach may not be sufficient, and even if it is, it's not likely to cope with environmental entropy very well.
If you can craft a gene[1] to express some particular phenotype (a big if), surely you can craft it to express itself without reliance on epigentic[2] chemistry.
[1] I understand gene to mean some ill-defined, not necessarily contiguous set of genetic sequences (DNA, RNA, and analogs) with an identifiable, particularized expression that effects reproductive (specifically, replicative) success. I think over time "gene" has been redefined and narrowed in a way to make it easier to claim to have made supposedly model-breaking discoveries.
[2] Some others on HN have made strong cases for why epigenetics isn't a meaningful departure from the classic genetic model; just a cautionary tail for eager reductivists who would draw unsupported conclusions from the classic model. See, also, note #1.
> Progress in AI is due to data and computational power advances.
I think you'd be surprised how much progress is also being made outside those two factors. It's sort of like saying graphics only improve with more RAM and faster compute. We know there's more to it than that.
In many cases, the cutting edge of a few years ago is easily bested by today's tutorial samples and 30 seconds of training. We're doing better with less data and orders of magnitude less compute.
I think it’s meant precisely in contrast to something like graphics, where the human element has obviously contributed alongside computational advances. “The Bitter Lesson”, basically. To the other point, aren’t computational advances the reason that it’s only 30 seconds of training?
But not towards AGI. We're just improving on narrow AI after recent breakthroughs thanks to the hardware being powerful enough and large datasets being available.
The point the poster above is trying to make is that given the same amount of data, improvements in technique is leading to significant improvements in accuracy.
An illustrative example comes from the first lesson in fastai's deep learning course: an image classifier that would have been SOTA as late as 2012/13, can be built by the hobbyist in like 30 seconds.
That said, I don't disagree that this is all narrow AI, at best.
Having access to cheap and scalable compute and storage should be helpful for AGI too. It doesn't solve anything but it does give more access to more people.
> even rodents can outperform state of the art models at general tasks
Rodents can't play Go or a lot of other humanly-meaningful tasks. We don't need to build an artificial cell. A cell is too many components that by blind luck happened to find ways to work together, this is as far from efficient design as can be. The same way we don't build two-legged airplanes, we don't need anything that's close to the wet spiky mess that happens in human brains. It's more likely that we have all the ingredients already in ML, and we need to connect them in an ingenious way and amp up the parallelism.
The problem with the analogy is that the car, by far, is not a general transportation device. Practically, most cars are solving a very constrained transportation problem: moving on roads that humans made.
We don't have anything remotely close to a wetware-enabled transportation device, something that can move on flat land, climb mountains, swim in bodies of water, crawl in caves, hide in trees.
Within the constrained problem, the machine exceeds humans. But generally, the wetware handles moving around much better.
Same with AI: in a constrained problem, the AI can excel (beat humans in chess and go). But I doubt we will see a general AI any time soon.
human AI also evolved by solving constrained problems, one at a time. Life existed before the visual system , but once this was solved it moved on to do other things. In AI we have a number of sensory systems seemingly solved: Speech recognition, visual object recognition, and we are closing to certain output (motor) systems: NLP text synthesis systems seem a lot like the central pattern generators that control human gait, except for language. What seems to be missing is the "higher-level ", more abstract kernels that create intent, which are also difficult to train because we don't have a lot of meaningful datasets. Or maybe , we have too big datasets (the entirety of wikipedia) but we don't know how to encode it in a meaningful way for training. It's not clear however that these "integrating systems" are going to be fundamentally different to solve than other subsystems. It certainly doesn't seem to be so in the brain, since neocortex (which hosts both sensory and motor and higher level systems) is rather homogeneous. In any case, it seems we 're solving problems one after another without copying nature's designs, so it's not automatically true that we need to copy nature in order to keep solving more.
> In AI we have a number of sensory systems seemingly solved: Speech recognition, visual object recognition,
Do you have examples of those systems which are competitive in general use rather than specialized niches? The cloud offerings from Amazon, Google, etc. are good in the specific cases they’re trained on but fall off rapidly once you get new variants which a human would handle easily.
There are many vision models where classification is better than human. I m not sure what you mean 'fall of rapidly'; they do fail however for certain inputs where humans are better. But we 're talking about models that contain 6 to 7 orders of magnitude less neurons than an adult brain.
It's also interesting in the context of how we build our technology in general: we constraint our environments just as much we develop tools that operate in them. E.g. much as cars were created for roads, we adapted our communities and the terrain around them by building roads and supporting infrastructure. A lot of things around us rely on access to clean water at pressure, which is something we built into our environments, etc.
AlphaZero has coded all the rules for the respective three games, they do a tree search and their neural network output layer has exactly n neurons for max(n) possible moves. Although it's impressive they don't teach it heuristics and strategies, it's a very specific task.
What about pigeons predicting breast cancer with 99% probability, rats learning to drive cars, monkeys building tools?
Rodents stand a bigger chance at learning Go than AlphaZero spontaneously building stone tools and driving cars.
You are talking about AlphaGo. AlphaZero was not given any prior knowledge of the game and is trained exclusively through self-play -- and it outperforms Monte Carlo tree search-based systems such as AlphaGo and Stockfish in chess 100-0 with a fraction of the training time.
AlphaZero is also capable of playing Chess, Shogi and Go at a super-super-human.
You can view these as optimized pattern recognizer regexes. You start with a blank fully connected graph and it eventually converge on a useful function. That graph has many paths encoded in it that represents specific optimal game play.
As impressive as AlphaZero surely is, I don't think it ever got a proper comparison to Stockfish. It was running on a veritable supercomputer while Stockfish was running in a crippled mode on crippled hardware.
Not working in this area but the abstract of the AlphaZero paper [0] seems to disagree about your /any prior knowledge/ point: "Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi (Japanese chess) as well as Go, and convincingly defeated a world-champion program in each case."
This is my point exactly. The model is trained without any prior domain knowledge at all. It only has access to a game world where the constrains in the world is a representation of the game's rules.
The natural environment encodes "all the rules" for real animals, too. You need some constraints or else there is nothing to be learned. One could say that every survival task is also specific , but is a slight variation of previously learned one.
> pigeons predicting breast cancer with 99%
pigeons contain 340M neurons (with dendrites and all, giving them higher computational capacity than ANN units).
> Rodents stand a bigger chance at learning Go
They probably don't ; probably because they can't understand the objective function and their brain capacity is limited
Scientist have just recently taught rats how to play hide-and-seek for fun. Other scientists have found out that slime mold will model the Japanese railroad system. I wouldn't be surprised if rodents (plural) instinctively have a go strategy once someone figures how to make an analog game for them.
its probably safe to assume that even if rodents are behaviorally trained to follow complex rules, they are mostly pattern-matching, and are lacking higher-level abstraction and communication models like humans do. If they did they would at least attempt to communicate with us, like we do with them. In such a case, an elephant that plays go by patternmatching is no different from a neural network that learned by patternmatching
> even rodents can outperform state of the art models at general tasks.
Rodents? Try insects [1]. In the late 40s and early 50s, when neural networks were first explored with great enthusiasm, some of the leading minds of that generation believed (were convinced, in fact) that artificial intelligence (or AGI in today's terms) is five/ten years away; the skeptics, like Alan Turing, thought it was fifty years away. Seventy years later and we've not achieved insect-level intelligence, we don't know what path would lead us to insect-level intelligence, and we don't know how long it would take to get there.
[1]: To those saying that insects or rodents can't play Go or chess -- they can't sort numbers, either, and even early computers did it better than humans.
>Something about predatory nature of both insects seems to tune up their intelligence.
Yep. To be a predator, you need to outwit your prey and think fast, so it's thought to be a natural INT grinder. `w´
Presumably, this could drive up the INT of prey too, but maybe it's cheaper to just be faster/harder to see? But you can't be THAT hard to see, and the speed only saves you in failed ambushes, so planning successful ambushes continues to reward the INT of predators (unless they just enter the speed arms race, like cheetahs or tiger beetles).
Speaking of Portias and smarts, I'm just going to recommend "Children of Time" here (and its recently released sequel, "Children of Ruin"). It's a story of a future where humans accidentally uplifted jumping spiders instead of monkeys, and goes deeply into how the minds, societies and technology of such spiders would be fundamentally different from our own.
> [1]: To those saying that insects or rodents can't play Go or chess -- they can't sort numbers, either, and even early computers did it better than humans.
They probably can, internally; they just can't operate on tokens we recognize as numbers explicitly. For a computer analogy, take Windows Notepad - there's probably plenty of sorting, computing square roots and linear interpolation being done under the hood in the GUI rendering code - but none of that is exposed in the interface you use to observe and communicate with the application.
Computers still do that much better -- there's no way an insect, or a mammal, brain internally sorts ten million numbers -- and even much better (at least faster) than humans. My point is only that the fact computers can do some tasks better than insects or humans is irrelevant, in itself, to the question of intelligence.
Having no clue is not something to be proud (or ashamed) of.
> I’m sorry to say a bunch of software engineers aren’t gonna crack this one.
Doesn’t sound like you’re at all sorry, it sounds like you’re thrilling in putting these uppity tryhards in their place for daring to attack something you hold sacred.
This reminds me of a interesting armchair moral dilemma: Assume we have the tech to replicate/simulate a biological brain. Now say we want to study the effects of extreme pain/torture etc on the brain. Instead of studying living animals or humans we'd just simulate a brain, and simulate sending it pain signals and see what happens.
But, if this is a 100% replicated brain, doesn't that mean its suffering is just as real as a real brain's suffering, and therefor just as cruel? And if not, what's the difference?
Fwiw, after a certain amount of pain, brain "transcends it": everything disappears, there are some curious colors here and there, but there is no pain. Experienced that during an in ear infection.
> 2. Ion channels may or may not be affected by quantum effects.
In a sense, everything is affected by quantum effects. However, with neurons, they are generally large enough that quantum effects do not dominate. Voltage gated channels are dozens to hundreds of amino-acids long. Generally, there are hundreds to millions of ion channels in a cell membrane and the quantum tunneling of a few sodium ions in or out of the cell will generally not affect gestalt behavior of the cell, let alone a nervous system's long term state. Suffice to say, ion channels are not dominated by quantum behavior.
Largely, we have the building blocks to replicate neurons (as we currently understand them) in silico. However, as is typical with modeling, you get out what you put in. Meaning that how you set your models up will mostly determine what they do. Setting your net size, the parameters of you PDEs, boundary values, etc. are the most important things.
Now, that gets you a result, and it's likely to take a fair bit of time to run through. To get it up to real time the limiting factor really ends up being heat. Silicon takes a LOT of energy as compared to our heads, ~10^4 more per 'neuron'. If we want to get to real time, we're gonna need to deal with the entropy.
> 1. Biological brains are non-differentiable spiking networks much more complicated than backpropagated ANNs.
Actually it's not so obvious that the brain is not differentiable. If you do a cursory search, you'll find quite a lot of research into biologically plausible mechanism for backpropagation. Not saying the brain does backprop, we just don't know and it's not outside of the realm of plausibility
This is so vague as in borderline sarcasm. Okk, it’s AGI but really, what is it? Reinforcement learning? Combining symbolic AI with modern advances? Deep learning theory? AGI is a vacuous term. And no one knows which path would lead to something similar to human intelligence. That itself is a matter of research.
The things you all listed are possible pathways to the goal. All he said is he wants to start working toward the goal. This guy is serious business, he's never been one to spew BS. He's a real deal, no bullshit computer scientist with leagues of novel accomplishments under his belt.
I wouldn't bet against him when he sets his mind to something.
Which is presumably the research he intends to do over the next few months? E.g. reading around the subject to determine where best to focus his efforts.
he is 49 this year, considered as one of the most genius programmer on earth. There are professors still doing real work at 90+ year old(yes, the UT professor goodenough for Nobel prize), John has a long way ahead, best luck!
i've heard he was pretty good, i didn't realize he was this widely respected and admired. I remember him for that fast inverse square root hack but that's about it.
931 comments
[ 4.7 ms ] story [ 241 ms ] thread>Congratulations on the new project, and may your hubris not doom us all.
> I am going to be going about it “Victorian Gentleman Scientist” style, pursuing my inquiries from home, and drafting my son into the work.
Which to me reads like part time work on Oculus, part time work on this AGI project. If it is with Facebook it isn't at all clear from the post (plus I'd assume it would be accompanied by marketing copy in that situation).
I will still have a voice in the development work, but it will only be consuming a modest slice of my time.
As for what I am going to be doing with the rest of my time: [...] For the time being at least, I am going to be going about it “Victorian Gentleman Scientist” style, pursuing my inquiries from home, and drafting my son into the work.
AGI needs a type system...
I hope I'll achieve AGI before him but it's nice to know there's some real competition! (because, reader, there are almost 0 researchers seriously trying to achieve AGI in a not totally bullshit way. Only opencog and Cyc comes to mind).
/ok, maybe his project falls under "total bullshit"...
He claims to have invented a program that is a mind, originally written in Forth, translated by others to many other languages, etc. He has published the code of this program in a form of "open source", so you can easily find it if you dig enough.
He's widely considered to be a crank. That said, the line between genius and madness can be mighty thin, and what side he lands on is anyone's guess, but most put him well over the line into madness, for whatever its worth.
My own opinion?
Well - looking at his work from purely the modern understanding and research into ML/AI - that is, deep learning and such - his work would be considered pointless, probably worse than Eliza as to its contributions to the field.
But as someone who has read a lot of other works (for for and against) the idea of AGI, artificial consciousness, theory of mind, etc - his work at a certain level has echos with some of that work. Still probably a dead end, but at the same time, there's some interesting concepts within his code and theories (he's self-published a book on it, too - you can find it on Amazon - he also has it for free on his github and it can be found elsewhere).
I guess I still put him in crank territory, but not in the abusive crank arena, more in the "doing his own thing, but being a bit evangelical about it" - relatively harmless.
His work is not as amazing as TempleOS, imho, but there's a similar mind behind it (though comparing it with that operating system is maybe an unfair, possibly orthogonal, comparison).
I won't say or reveal more (but I've written enough for you to figure it out) - he tends to monitor tons of forums and if he thinks he's being "summoned", he'll spam the forum with his writings and theories. It got him "perma-banned" from more than one newsgroup back in the day...
Contrary to popular belief, both openai and deepmind have zero roadmap and specification of a cognitive architecture, not even a semantic parser.
https://news.ycombinator.com/item?id=21542453
You might just begin by learning the list of NLP tasks and how good are the state of the art at it. The cognitive architecture that needs to be created to achieve AGI will one way or another be a composition of said tasks, which are the primitives.
You can discover such a taxonomy here: https://github.com/sebastianruder/NLP-progress/blob/master/R...
Also you might be interested by learning logic as a big task is to translate natural language into queryable, logical forms.
I do this on my free time so I'm not productive, but I have designed an intermediary language (IR) for natural languages that seems very promising.
Either what you actually mean by "syllogism checker" is extremely specific and unpractical, or this is 100% BS.
For example, OpenCog is an implementation of the classic cognitive architecture and its about as traditional and far from "total bs" as you can get in AGI.
I have never heard anything to back up the insults against Goertzel.
My brain bit on that remark; would you care to elaborate?
For anybody who hasn't played an untethered VR experience, I highly recommend it. It makes a world of difference with games like Echo combat and Beatsaber. Tons of fun. It reminds me of the first time I played wii bowling.
I've long thought that the issue with VR is a conceptual one, not a technical one and maybe that frustration comes from there. "Running forward" is an unsolved problem in room scale VR. For a seated experience, you're basically back to a neat display gimmick + accurate hand tracking.
Any real solutions need, on the one side, real-world physical constructions (think running threadmills) that soon hit holodeck-level limitations and, on the other, software that actually benefits from the real technology VR brings to interactive media: super accurate hand- and head-tracking. The first gets impractical/impossible soon, the second limits development to a few niche genres: Shooting ranges, cockpit sims, dance/party games and some vague "experiences" where the actual tech is pretty much ignored and you just say "but it feels so immersive!" (honestly, it does work for horror games!). It's basically motion controls 2.0.
The only place I could see the technology shine is, oddly enough, AR. It has way less mainstream hype to it but it makes much more sense because you actually benefit from the tracking of your real-world movement: You're still a part of it! The holo-lens demos that pop up on youtube might seem clumsy, but I can totally see a use case for replacing physical monitors with arbitrarily sized and positioned displays you can virtually move in any office space. There's rumors of Apple working with Valve on AR tech. If there's any technology that could follow the smart phone, AR is my bet. I'm honestly surprised Carmack didn't move in that direction rather than deciding to become a general AI guru.
This was the (loud) subtext for me, too.
Carmack doesn't strike me as the type of person to walk away from a problem lightly. Given how many problems remain unsolved when it comes to VR, I wonder if he's just admitted that it will never be the endgame he wanted it to be.
Cash-wish, he's obviously sitting pretty. Better off spending your remaining years working on something you can make a meaningful contribution towards.
Wouldn't it make more sense for him to join a cutting-edge team, like DeepMind or OpenAI?
This is exactly the experience of most teams I've spoken with, be they students or businesses, for all the pre-production phase. You simply can't and shouldn't spend on costly AI infrastructure before you've nailed your solution; in fact any kind of infra not just AI.
What you do is rent some cloud to power quickly through your tests — better have 10x worth of big Nvidia GPUs over 2 weeks than buy 1 or 2 max yourself and wait 5-10x more time — not even factoring that setting up clusters of GPU and running such workflows consistently over days, weeks requires pretty deep sysadmin/hardware knowledge and experience; it took me two years to really master that non-problem part on my home server (but now it's a skill I have so that was worth it, but certainly set my research and learning back by as much time).
Besides, there's a time when the familiarity, safety and general comfort of home simply can't be beat. Notwithstanding pool tables and free soda, lol.
He's openly talked about his work ethic in a bunch of places. He's the type of guy who after a life time of coding calculated he's 100% efficient up until 13 hour work days and then he drops off[0]. Although he did mention working those long hours is often best working on multiple things instead of 1 topic but maybe with AGI there's a bunch of different avenues to explore.
[0]: https://www.youtube.com/watch?v=udlMSe5-zP8&t=4773
You are bound by the amount of data and computational resources you have at your disposal. Neither are tied to man hours. You can stay up all night for days waiting for your model to train, and it will do you no good.
Unfortunately that could never happen because of the SEC.
I wouldn't be surprised if the opposite was true, at least with the theory part. AI didn't really go anywhere for decades, because people focused too much on theory.
Otherwise, there's a lot of data and hardware at your disposal, even from the comfort of your home.
> Wouldn't it make more sense for him to join a cutting-edge team, like DeepMind or OpenAI?
You mean they guys that are training with videogames that people like John developed?
Substitute math for anything you want.
Everything is easy in hindsight.
And that's without mentioning that the specific result you are talking about is an infinitesimal progress when compared to "AGI".
> It's the kind of problem that requires incredible amounts of data, hardware, and theory to make any progress.
So I point out a recent example of progress (of which the sequence of fundamental insights is nearly always incremental), where the theoreetical insight was a theoretical derivation, which could be and probably was derived on paper / blackboard, as a direct counterexample: it does not require "incredible amounts of [...] theory".
Why would you compare this with performing manual backpropagation?
>Everything is easy in hindsight
Every breakthrough is non-trivial, else it would not have been a breakthrough, and yet the breakthrough itself can be a relatively simple calculation...
The concept of "AGI" is undefined and virtually worthless to me. There is only non-trivial insights, i.e. theorem and proof.
Applying intelligence to artificial intelligence has happened before https://en.wikipedia.org/wiki/Shakey_the_robot. One of the contributors to Shakey was Alfred Brain.
John is the perfect representation of what is wrong with peoples attitude toward AGI. aloof and naive.
Because that's really where the line of human history will be drawn if AGI and above becomes real. AI safety, how advanced we are in it, will directly map to civilization's progress or endangerment as a result of AI.
Edit: this is already true with regards to "psychological safety" from undue influence or outright manipulation with motive (usually financial) by current "ANI" algorithms (newsfeeds, "recommendations", ads, etc). It's a real topic that reduces to human psychological freedom, freewill. It's a BIG topic.
Probably? I do not think that word means what you think it means... or else I don't think the balance of probability lies where you think it does.
This dismal, dimwitted "advanced" filter and sort industry has recently started training all their employees in and vomiting all over every consumer with is nothing, worthless, and at best lunacy.
All it takes is patience, know-how, and insight. For that, Carmack fits the bill.
I know of several million species that would strongly disagree. Especially if AGIv1 decides to tune the genetics of say most mammals to append 'sapient' to the end of their species name.
Perhaps more constructively consider that AGI is simply the next iteration of 'humanity', yea sure the old versions are redundant anarchisms and apart from some living reserve specimens functionally extinct, but nobody cares as you can sim one up at almost no cost.
https://news.ycombinator.com/newsguidelines.html
Computer science now occupies the place physics once did, in its impact on moving the world forward.
Best of luck to him, I look forward to seeing what he produces!
If you have a strong stance about the evils of software development then write something of substance and post it. I am sure it will get plenty of discussion if there's any merit to it.
Can you back that up? Here's the comment:
> You guys really get high on your own supply don't you? It's a toss-up these days whether something from computer science or a programmer will either enhance or erode the human experience and quality of life.
"getting high on your own supply" isn't such a giant insult that you can simply "make the dozen full" and claim whatever you want about the comment.
> I am sure it will get plenty of discussion if there's any merit to it.
Then you're sure of something that is demonstrably false, and one-off low effort sophistry like this offers nothing of value. There is no hard connection between the merit of something and HN's ability to discuss it. Take this story sticking to the top for probably over 24 hours, basically an announement to spend more time with his son, and that's it, while this got sunk off the front page yesterday: https://news.ycombinator.com/item?id=21527622
No one here is claiming that software development doesn't make fat stacks and viable businesses. It just happens to do it by perverting humanity. Moving us further away from our ideal environment.
There was a 50/50 chance 100 years ago that discoveries in chemistry or physics were immediately weaponized. I'd say GP's analogy still holds, yeah?
Nature has brute facts that can only be discovered through observation. I would be surprised to see evidence that an intelligence, no matter how smart, could reason to the fundamental properties of neutrinos without massive physical real world experiments and piles of observational data.
Physics gave us airplanes and bombs to drop from them.
Biology and chemistry gave us pharmaceuticals and pollutants.
You can do this for any field of study really.
You are making a common mistake of assuming that just because someone is good at something like programing computers the same skill would translate identically to a completely different domain.
If anything he is lucky to have been born in an era where his skill of programming computers could be put to use - otherwise his talents may have gone to waste, he may have ended up toiling fields his talent untapped and undiscovered, like that of millions before him.
And then he has the ability to power through learning what he needs to in order to build toward those solutions lightning fast.
He's the kind of person where, if you show him what you're working on and he doesn't understand it, you probably need to go back to the drawing board.
For what is worth computers are unique tools unlike any other tool that mankind has invented before - thus it is much harder to tell what other jobs would a good programmer excel at.
EDIT: I'll elaborate a bit. In my experience in both industry and academia I've witnessed numerous occasions when brilliant people would get things wrong, ignore a brilliant idea, follow a hopeless research direction, etc. etc. Authority matters, but _nobody_ is flawless.
Most programmers don't do that as their side project.
I see Carmack as a very(as in uniquely) talented Engineer. Usually, engineers are not the type who do very well in pure research topics. And AGI is certainly a pure research topic, since we don't have a clear leading us there. So while it's great to see he is interested in it, now sure if we should have any kind of expectation there.
1. Biological brains are non-differentiable spiking networks much more complicated than backpropagated ANNs.
2. Ion channels may or may not be affected by quantum effects.
3. The search space is huge (but organisms aren't optimal and natural selection is probably local search)
4. If it took ~3.8b years to get from cells to humans, how do we fast-forward:
* brain mapping (replicating the biological "architecture")
* gene editing on animal models to build tissues and/or brains that can be interfaced (and if such interface could exist how do we prevent someone from trying to use human slaves as computers? Using which tissues for computation is torture?)
* simulation with computational models outside of ECT (quantum computers or some new physics phenomenon)
Note: those 3.8b years are from a cell to human. We haven't built anything remotely similar to a cell. And I'm not claiming that an AGI system will need cells or spiking nets, most likely a lot of those are redundant. But the entropy and complexity of biological systems is huge and even rodents can outperform state of the art models at general tasks.
IMHO, the quickest path to AGI would be to focus on climate change and making academia more appealing.
I would imagine that a PDE may cover diffuse behaviors governing say, how learning happens mechanically, but it is almost certainly a language/representational barrier, the relationship between the structure of the animal mind, learning, and seemingly simple phenomena, like afterimages.
The molecules are arbitrary and the timescale doesn't matter.
>You assert that an area of physics or mathematics familiar to few neuroscientists solves a fundamental problem in their field. Example: "The cerebellum is a tensor of rank 10^12; sensory and motor activity is contravariant and covariant vectors".
So yeah, I feel that it's pretty fringe (as you suggested).
[0] https://web.archive.org/web/20170828092031/http://cns-web.bu...
[1] https://www.smbc-comics.com/comic/the-talk-3
So it is plausible that nature may have evolved to be affected by quantum effects.
The key, of course, is redefining life and intelligence as whatever the current state-of-the-art accomplishes. (Cue explanations that the brain is just a giant pattern matcher.) It makes drawing parallels and prophesying advancements so much easier. Of all our sciences, that's perhaps the one thing we've perfected--the science of equivocation. And we perfected it long ago; perhaps even millennia ago.
Gene expression is often tied to the environment the organism is in. Mere possession a gene isn't enough to benefit from it. Some expressions don't take effect immediately, but rather activate in subsequent generations.
Epigenetics is a whole equally large layer on top of this system. A single-focus approach may not be sufficient, and even if it is, it's not likely to cope with environmental entropy very well.
[1] I understand gene to mean some ill-defined, not necessarily contiguous set of genetic sequences (DNA, RNA, and analogs) with an identifiable, particularized expression that effects reproductive (specifically, replicative) success. I think over time "gene" has been redefined and narrowed in a way to make it easier to claim to have made supposedly model-breaking discoveries.
[2] Some others on HN have made strong cases for why epigenetics isn't a meaningful departure from the classic genetic model; just a cautionary tail for eager reductivists who would draw unsupported conclusions from the classic model. See, also, note #1.
I think you'd be surprised how much progress is also being made outside those two factors. It's sort of like saying graphics only improve with more RAM and faster compute. We know there's more to it than that.
In many cases, the cutting edge of a few years ago is easily bested by today's tutorial samples and 30 seconds of training. We're doing better with less data and orders of magnitude less compute.
An illustrative example comes from the first lesson in fastai's deep learning course: an image classifier that would have been SOTA as late as 2012/13, can be built by the hobbyist in like 30 seconds.
That said, I don't disagree that this is all narrow AI, at best.
Rodents can't play Go or a lot of other humanly-meaningful tasks. We don't need to build an artificial cell. A cell is too many components that by blind luck happened to find ways to work together, this is as far from efficient design as can be. The same way we don't build two-legged airplanes, we don't need anything that's close to the wet spiky mess that happens in human brains. It's more likely that we have all the ingredients already in ML, and we need to connect them in an ingenious way and amp up the parallelism.
We don't have anything remotely close to a wetware-enabled transportation device, something that can move on flat land, climb mountains, swim in bodies of water, crawl in caves, hide in trees.
Within the constrained problem, the machine exceeds humans. But generally, the wetware handles moving around much better.
Same with AI: in a constrained problem, the AI can excel (beat humans in chess and go). But I doubt we will see a general AI any time soon.
human AI also evolved by solving constrained problems, one at a time. Life existed before the visual system , but once this was solved it moved on to do other things. In AI we have a number of sensory systems seemingly solved: Speech recognition, visual object recognition, and we are closing to certain output (motor) systems: NLP text synthesis systems seem a lot like the central pattern generators that control human gait, except for language. What seems to be missing is the "higher-level ", more abstract kernels that create intent, which are also difficult to train because we don't have a lot of meaningful datasets. Or maybe , we have too big datasets (the entirety of wikipedia) but we don't know how to encode it in a meaningful way for training. It's not clear however that these "integrating systems" are going to be fundamentally different to solve than other subsystems. It certainly doesn't seem to be so in the brain, since neocortex (which hosts both sensory and motor and higher level systems) is rather homogeneous. In any case, it seems we 're solving problems one after another without copying nature's designs, so it's not automatically true that we need to copy nature in order to keep solving more.
Do you have examples of those systems which are competitive in general use rather than specialized niches? The cloud offerings from Amazon, Google, etc. are good in the specific cases they’re trained on but fall off rapidly once you get new variants which a human would handle easily.
What about pigeons predicting breast cancer with 99% probability, rats learning to drive cars, monkeys building tools?
Rodents stand a bigger chance at learning Go than AlphaZero spontaneously building stone tools and driving cars.
AlphaZero is also capable of playing Chess, Shogi and Go at a super-super-human.
[0] https://arxiv.org/abs/1712.01815
> pigeons predicting breast cancer with 99%
pigeons contain 340M neurons (with dendrites and all, giving them higher computational capacity than ANN units).
> Rodents stand a bigger chance at learning Go
They probably don't ; probably because they can't understand the objective function and their brain capacity is limited
Can't tell if sarcasm.
Rodents? Try insects [1]. In the late 40s and early 50s, when neural networks were first explored with great enthusiasm, some of the leading minds of that generation believed (were convinced, in fact) that artificial intelligence (or AGI in today's terms) is five/ten years away; the skeptics, like Alan Turing, thought it was fifty years away. Seventy years later and we've not achieved insect-level intelligence, we don't know what path would lead us to insect-level intelligence, and we don't know how long it would take to get there.
[1]: To those saying that insects or rodents can't play Go or chess -- they can't sort numbers, either, and even early computers did it better than humans.
They are creepy smart.
Something about predatory nature of both insects seems to tune up their intelligence. Of course it never hurts having the BBC tell your story either.
Yep. To be a predator, you need to outwit your prey and think fast, so it's thought to be a natural INT grinder. `w´
Presumably, this could drive up the INT of prey too, but maybe it's cheaper to just be faster/harder to see? But you can't be THAT hard to see, and the speed only saves you in failed ambushes, so planning successful ambushes continues to reward the INT of predators (unless they just enter the speed arms race, like cheetahs or tiger beetles).
They probably can, internally; they just can't operate on tokens we recognize as numbers explicitly. For a computer analogy, take Windows Notepad - there's probably plenty of sorting, computing square roots and linear interpolation being done under the hood in the GUI rendering code - but none of that is exposed in the interface you use to observe and communicate with the application.
Like what is language, what is intelligence? Some of the smartest linguists and philosophers would proudly declare they have no fucking clue.
Making Alexa turn on the lights or using Google Translate are cool party tricks though.
Idc how many Doom games ya made, but I’m sorry to say a bunch of software engineers aren’t gonna crack this one.
“to worship a phenomenon because it seems so wonderfully mysterious, is to worship your own ignorance” - https://www.lesswrong.com/posts/x4dG4GhpZH2hgz59x/joy-in-the...
Having no clue is not something to be proud (or ashamed) of.
> I’m sorry to say a bunch of software engineers aren’t gonna crack this one.
Doesn’t sound like you’re at all sorry, it sounds like you’re thrilling in putting these uppity tryhards in their place for daring to attack something you hold sacred.
But, if this is a 100% replicated brain, doesn't that mean its suffering is just as real as a real brain's suffering, and therefor just as cruel? And if not, what's the difference?
Yes, it does.
This reminds me of the idea that free will doesn't exist, but that we have to act as if it were.
So by analogy to that, maybe the AI isn't really suffering, but you have to act as if it were.
More food for thought:
Some surgery blocks memory but can be incredibly painful. Do we need to worry about that? Is the suffering that the brain can not remember "real"?
In a sense, everything is affected by quantum effects. However, with neurons, they are generally large enough that quantum effects do not dominate. Voltage gated channels are dozens to hundreds of amino-acids long. Generally, there are hundreds to millions of ion channels in a cell membrane and the quantum tunneling of a few sodium ions in or out of the cell will generally not affect gestalt behavior of the cell, let alone a nervous system's long term state. Suffice to say, ion channels are not dominated by quantum behavior.
Largely, we have the building blocks to replicate neurons (as we currently understand them) in silico. However, as is typical with modeling, you get out what you put in. Meaning that how you set your models up will mostly determine what they do. Setting your net size, the parameters of you PDEs, boundary values, etc. are the most important things.
Now, that gets you a result, and it's likely to take a fair bit of time to run through. To get it up to real time the limiting factor really ends up being heat. Silicon takes a LOT of energy as compared to our heads, ~10^4 more per 'neuron'. If we want to get to real time, we're gonna need to deal with the entropy.
Actually it's not so obvious that the brain is not differentiable. If you do a cursory search, you'll find quite a lot of research into biologically plausible mechanism for backpropagation. Not saying the brain does backprop, we just don't know and it's not outside of the realm of plausibility
I can't tell if he was serious about that comment or not... Considering he builds rockets with free time, it could go either way.
Nuclear fission shocks you but not AGI?
I wouldn't bet against him when he sets his mind to something.