“Terminator: The man most directly responsible is Miles Bennett Dyson.
Sarah Connor: Who is that?
Terminator: He’s the director of special projects at Cyberdyne Systems Corporation.
Sarah: Why him?
Terminator: In a few months, he creates a revolutionary type of microprocessor.
Sarah: Go on. Then what?
Terminator: In three years, Cyberdyne will become the largest supplier of military computer systems. All stealth bombers are upgraded with Cyberdyne computers, becoming fully unmanned. Afterwards, they fly with a perfect operational record. The Skynet Funding Bill is passed. The system goes online on August 4th, 1997. Human decisions are removed from strategic defense. Skynet begins to learn at a geometric rate. It becomes self-aware at 2:14 AM, Eastern time, August 29th. In a panic, they try to pull the plug.
Sarah: Skynet fights back.
Terminator: Yes. It launches its missiles against the targets in Russia.
John Connor: Why attack Russia? Aren’t they our friends now?
Terminator: Because Skynet knows that the Russian counterattack will eliminate its enemies over here.”
Do you want Terminators? This is how you get Terminators.
But in all seriousness, this seems the most logical next step. AIs are getting so complicated that you need an AI to understand it. We just need to make sure they don't get so black box that we trust the outputs blindly.
> We just need to make sure they don't get so black box that we trust the outputs blindly
Black box is probably the only way we'll ever get AGI.
Our human brains are so black box that we can only accept their outputs blindly. We try to poke and prod the human black box as best we can, with magnets, sound waves, and light waves.
But it still generates useful output all the time, so we assume it's pretty good and generally let it be.
This is fair. We trust our own black boxes every day. You have a good point.
But on the flip side, even today, a large part of the population doesn't trust people whose black boxes work faster than theirs. Now imagine a new black box that's 1000 times faster.
In fairness to all of us humans, no. We do not blindly trust our brains. This explicitly how science came about, and this is also how entire branches of philosophy came about.
We do blindly trust our brains all the time. Whenever a brain or a group of brains tells us we need chemotherapy or to start a war, with barely the slightest understanding of how the brain is wired or based on which inferences it would have come to that conclusion, we believe it.
Obviously we believe only those brains that were primed with enough training data and have a decent track record of giving useful answers and making accurate-enough predictions.
But we still rely on this "blindly" in the absence of their source code or even any understanding about how or why they think, or why some collections of cells reason and demonstrate intelligence, while most don't.
We currently hold machine brains to a much higher standard of transparency and comprehensibility than we do human brains.
Human brains can't even tell us much about themselves or show any logging, because they barely know anything about how they work or where ideas or connections come from either.
In general, no. If we blindly trusted our brains, we would not require things like “another set of eyes,” or the rigors of reproducibility, or business processes/protocols, or a million other things. Further more, this lack of trust in the matter between our ears is observable in the course of daily human life when such questions are asked as “why did I say that?” or “why did I do that?”. Blind trust would obviate the need for such questions. As for the supposed trust in authority that you describe, that is not blind trust in the brain. Trust in authorities is actually quite low at the moment, and was never really blind. Every election in the USA, half of the country doesn’t trust the people elected, and possibly even the election itself. More over, War is no longer declared, and people never blindly trusted the government on war anyway, hence all of the protests of all of the wars. Trusting scientists is clearly not a universal thing either, hence the need for reproducibility as well as very large minorities of people not trusting rather obviously provable stuff (like landing on the moon where nearly any ham radio operator could tell you we did). Even if trust were not low, trusting an authority is built of many things. Brain washing, propaganda, protocol/process, committees, culture, tradition, instinct, and many more things all work together to form obedience to authority… trust in the brain isn’t much of a requirement in obeisance to authority figures.
The paper is well-written, and I would recommend reading its abstract and introduction over the OP.[a] The big idea is that a graph neural network (GNN) can be trained to predict the weights of any other neural network for performing a task, given the architecture as an input.
For example, if we give the trained GNN a ResNet-50 architecture, the GNN predicts the trained weights in one forward pass, and voilà, we have a ResNet-50 that is now ready for use -- and you can always finetune it to get better performance out of it.
Remarkably, the prediction of weights works for previously unseen model architectures too. Design your own model, run its architecture by the GNN to predict the trained weights, and voilà, you have a model that is now ready for use -- and you can always finetune it to get better performance out of it.
Very cool!
--
[a] https://arxiv.org/pdf/2110.13100.pdf -- Abstract: "Deep learning has been successful in automating the design of features in machine learning pipelines. However, the algorithms optimizing neural network parameters remain largely hand-designed and computationally inefficient. We study if we can use deep learning to directly predict these parameters by exploiting the past knowledge of training other networks. We introduce a large-scale dataset of diverse computational graphs of neural architectures – DEEPNETS-1M– and use it to explore parameter prediction on CIFAR-10 and ImageNet. By leveraging advances in graph neural networks, we propose a hypernetwork that can predict performant parameters in a single forward pass taking a fraction of a second, even on a CPU. The proposed model achieves surprisingly good performance on unseen and diverse networks. For example, it is able to predict all 24 million parameters of a ResNet-50 achieving a 60% accuracy on CIFAR-10. On ImageNet, top-5 accuracy of some of our networks approaches 50%. Our task along with the model and results can potentially lead to a new, more computationally efficient paradigm of training networks. Our model also learns a strong representation of neural architectures enabling their analysis."
So this is for quickly training different architectures for the same task? Which is very neat, but it makes the headline seem a lot more like click-bait (I'm not sure what I expect these days...)
It's kind of like a more advanced version of hyper-parameter tuning which has been done in more primitive forms for a long time. But this approach definitely seems powerful.
> So this is for quickly training different architectures for the same task?
It's for instantly training/pretraining any architecture for one or more predefined tasks (e.g., you could have a GNN for visual tasks, a GNN for NLP tasks, etc.), but you could also have a multi-domain GNN which accepts two inputs: an architecture and a domain.
Can anyone enlighten me how you'd make use of this practically though? It seems like you need like a dataset of a million models to train this beast (all of which trained on a specific dataset). Yeah it can figure out the weights for a new model but you had to train 1 million models first to be able to even do that. Am I missing something?
Practical use: Once you have GNNs that predict the trained weights of headless models for vision, NLP, and other domains, you can test new architectures in all those domains much faster with fewer compute resources.
Research use: Can we figure out how and why these GNNs can predict trained weights for previously unseen architectures without actually having to train the models? Can we simplify the GNNs? Can we use the knowledge gained from studying these GNNs to speed-up conventional training?
> Practical use: Once you have GNNs that predict the trained weights of headless models for vision, NLP, and other domains, you can test new architectures in all those domains much faster with fewer compute resources.
Aha. Using this as an accelerator for Neural Architecture Search would indeed be very neat.
You still need to train 1 million architectures though before you can start the search (dataset inflexible)…
I suppose there’s some value in possibly finding radically different networks (though I wonder if it’d fare well in such outlier regions). Perhaps if/since the GNN model is also differentiable you could invert it to maximize some compute/quality trade off point.
But like it already exists. That's a sunk cost. Presuming it actually works and gives useful weights, if it lowers training time of novel models by 10% or something, that's dollars saved in the long run.
If you work on a particular problem (lets say because it is your startup's object of research) you very likely are testing a large number of architectures on your dataset.
With this you could train a GNN on the various models you tested to speed-up training new ones.
As far as I can tell, SOTA ("state of the art") is 99% on CIFAR-10 [1]. The first entry in wikipedia, for 2010, shows 21.1% error rate, so 78.9% accuracy. Which is to say a 60% accuracy rate is far from useful to put it mildly.
Essentially, with that low an accuracy rate, I don't think there's reason to think the predictions are meaningful in a practical sense. But maybe there's something I'm missing in accuracy description.
Same goes for ImageNet: These GNNs are getting only 50% top-5 accuracy, well below SOTA.
But I find that figure impressive: It implies that I can concoct a newfangled architecture for visual recognition, put it through one of these GNNs, and begin training at ~50% top-5 accuracy. Also, AFAIK this is the first effort of its kind; I would expect the figures to improve over time, as usual.
Very cool and sensical when you pause to think about it. While different architectures might have different components, they are also likely to reuse building blocks (ie: the first layers of convolutional neural networks are famously very similar from one problem to another).
In essense this detects those building blocks and plugs the known weights in for you.
Well, this reminds me of a novel by David Gerrold "When HARLIE Was One (Release 2.0)" The book even mentions a "hyperstate processor" and, without giving to much away, a Graphical Omniscient Device (G.O.D). The similarities are pretty interesting.
If we ever achieve AGI, my guess is that something akin to this approach is how it will probably be accomplished. I don’t think the human brain will really be able to intuitively grok however AGI works, so I see an iterative process where we invent a program that tries to find a better program to accomplish some goal. Then we feed that program back into itself, and it recursively finds a better program finder. It seems like if we can just get past that critical threshold where the recursive process builds on itself rather than fizzles out due to resource constraints, then progress should accelerate quickly from there.
I’m also thinking along these lines. With little to no knowledge of the domain (other than being human) a lot of life seems to come down to referencing which subsystem or set of subsystems to use in a given situation and training them over time. Like yes - we need the ‘make small talk with checkout operator’ skill in this situation
Hard agree with this. Humans are always going to insert human biases into the systems we design, so I think the only way we can really get past those biases is by going several layers of abstraction deep. I actually suspect that the first AGI will behave eerily organically when we take a look under the hood. Evolution is the ultimate optimization function, and it made organic brains work a certain way based on a cost function. I'll bet we'll see patterns in AGI that remind us of biological brains, but with stark, alien components mixed in and structures reorganized to be more efficient.
All in all I'm just incredibly excited to see how this line of research plays out and desperately want to get involved in it myself. I was actually considering learning JAX just the other day specifically so I could look more into the topic, but I'm lazy so I never got around to it.
> and it recursively finds a better program finder
This has been bugging me for a while. I can't articulate it well but I feel there's a hole in the logic. What exactly is the "program finder" finding? A better version of the program at finding... what exactly? You need a stop condition for the recursion.
Serious question. Can we build an AI that builds the AI that builds the AI? What are the theoretical consequences of going higher and higher up the build chain?
This is something I'm actually, genuinely really interested in. In fact, just the other day I was thinking about whether you could use GNNs to accomplish something like this -- so I was understandably pretty hyped to see this post on HN.
I've been struggling to find the right terms to Google to find out more about the field, but it looks like "hypernetwork" was what I was looking forward. I highly, highly doubt that the first AGI is going to be designed by hand by humans -- I'd be willing to bet money that the first AGI is going to be recursively designed by multiple layers of constructor AIs just like what you're proposing.
Incredibly exciting idea IMO, and I'm confused why there doesn't appear to be more interest in it.
>I highly, highly doubt that the first AGI is going to be designed by hand by humans
Regular AI isn't even done by hand anymore. We just pick data and learning parameters. At the current state we'd be picking higher and higher categories for learning. At the lowest level we pick data about cats to write something that recognizes cats. At a higher level we pick data about things that recognize things and so on... and so on.
How does extremely high level training data effect the low level accuracy of the AI at the lowest level? I would presume that it would effect it negatively as technically all AI is estimation. So estimations on estimations are less accurate.
Hard to overstate how exciting I find the fields of hypernetworks and metalearning. For good or for evil, this I feel is almost certainly how we create AGI. GNNs make perfect sense as a method of accomplishing it too -- after all, neural networks are just graphs anyways.
I don't believe AGI is going to conform to our rigid, symmetrical, boxy human-made architectures with their nice, neat flow patterns and powers-of-two factors everywhere -- those architectures work great for what they're designed to do, but achieving sentience/true intelligence just really doesn't feel like it's going to come out of a feedforward MLP, or even a transformer (in my opinion).
The most complicated and capable neural networks we know of are the organic, messy, squishy, yet organized electric meat lumps in our own skulls, produced by billions of years of being exposed to the same loss function over and over and over, but with unlimited ways to adapt in order to minimize that loss function. Nature makes awesome, fractally, insanely complex networks that we could never design by hand. I think it's only sensible to assume that AGI will work similarly.
Of course, this is all just armchair speculation. I'm not an expert, just an enthusiastic data scientist who thinks these next twenty years in AI are going to be a heck of a freaking ride.
If we do achieve AGI through a setup which is not only neural network based, but where even the architecture by which the nodes are connected is beyond our understanding, then, that sounds very bad for interpretability and therefore for safety.
Sure. I'm all for safety, but I also think that unsafe AGI, like the atom bomb, is an inevitability. It's not a matter of if, it's a matter of when we invent it and release it. The future is already murky but it's about to get a heck of a lot more so -- sentient AIs could be the thing that exalts humanity or the thing that destroys us. Break out the popcorn, because I'd wager we're going to see one of those outcomes in the next century.
If we achieve AGI, then the AGI should be able to produce a formal verification proof of its own safety, similar to how humans use produce formal verification proofs for critical software using automated theorem provers today.
Well, to have a formal proof of its own safety, we first would need a correct formal definition of what it means for it to be safe (or, rather, a formal definition of a condition which is truly a sufficient condition for it to be safe. It needn't be a necessary condition.)
But, I suppose arriving at such a formal definition doesn't really require that the thing being described be particularly amenable to our understanding.
However, I'm not sure that "if we ran it, if it is safe, it could prove itself to be safe" is sufficient to address the concerns.
If it isn't safe, then running at all may spell doom, so, "if it is safe, it could demonstrate that after we turn it on" doesn't really address that, because it doesn't give us any assurance before we turn it on.
Now, if we started with something with substantially sub-human overall intelligence, and which wasn't really agent-y (so that, if it was a little unsafe, it would at least not be catastrophically unsafe), but which was more equipped to formally prove things about itself and potential modifications of itself than humans are equipped to formally prove things about it, then we could task that thing with formally proving safety properties about itself, and of also proving safety properties about its successor, and do that before running its successor, and iterate this process to produce increasingly intelligent programs and perhaps also eventually agentic ones, while always having a safety proof of each before we run it..
But, I'm not really sure how plausible this route is? Like, even assuming we do reach AGI, safe or unsafe, I'm not sure this is a plausible route of getting there safely.
Well, we don't have anything like a coherent understanding of the architecture of biological general intelligence either, so at least this won't be a particularly new state of affairs.
Just kidding: After skimming the article, this looks like necessary progress. I myself (as a layman) had assumed this is already how they did it - although I'm sure there's a lot more to it than that.
“One can, in theory, start with lots of architectures, then optimize each one and pick the best. “But training [takes] a pretty nontrivial amount of time,” said Mengye Ren, now a visiting researcher at Google Brain. It’d be impossible to train and test every candidate network architecture. “[It doesn’t] scale very well, especially if you consider millions of possible designs.””
——-
If you wanted to find best architecture in order to maximize accuracy, why not just train a model to predict accuracy (not parameters) given architecture and then optimize over the model?
This seems similar to optimizing any expensive black box function. Fit a cheap approximation (i.e., surrogate model) and then optimize over cheap model.
58 comments
[ 4.3 ms ] story [ 192 ms ] threadSarah Connor: Who is that?
Terminator: He’s the director of special projects at Cyberdyne Systems Corporation.
Sarah: Why him?
Terminator: In a few months, he creates a revolutionary type of microprocessor.
Sarah: Go on. Then what?
Terminator: In three years, Cyberdyne will become the largest supplier of military computer systems. All stealth bombers are upgraded with Cyberdyne computers, becoming fully unmanned. Afterwards, they fly with a perfect operational record. The Skynet Funding Bill is passed. The system goes online on August 4th, 1997. Human decisions are removed from strategic defense. Skynet begins to learn at a geometric rate. It becomes self-aware at 2:14 AM, Eastern time, August 29th. In a panic, they try to pull the plug.
Sarah: Skynet fights back.
Terminator: Yes. It launches its missiles against the targets in Russia.
John Connor: Why attack Russia? Aren’t they our friends now?
Terminator: Because Skynet knows that the Russian counterattack will eliminate its enemies over here.”
But in all seriousness, this seems the most logical next step. AIs are getting so complicated that you need an AI to understand it. We just need to make sure they don't get so black box that we trust the outputs blindly.
Black box is probably the only way we'll ever get AGI.
Our human brains are so black box that we can only accept their outputs blindly. We try to poke and prod the human black box as best we can, with magnets, sound waves, and light waves.
But it still generates useful output all the time, so we assume it's pretty good and generally let it be.
But on the flip side, even today, a large part of the population doesn't trust people whose black boxes work faster than theirs. Now imagine a new black box that's 1000 times faster.
Obviously we believe only those brains that were primed with enough training data and have a decent track record of giving useful answers and making accurate-enough predictions.
But we still rely on this "blindly" in the absence of their source code or even any understanding about how or why they think, or why some collections of cells reason and demonstrate intelligence, while most don't.
We currently hold machine brains to a much higher standard of transparency and comprehensibility than we do human brains.
Human brains can't even tell us much about themselves or show any logging, because they barely know anything about how they work or where ideas or connections come from either.
Now you have two problems.
AIs are already effectively black boxes to most people, and too many already trust AIs blindly.
For example, if we give the trained GNN a ResNet-50 architecture, the GNN predicts the trained weights in one forward pass, and voilà, we have a ResNet-50 that is now ready for use -- and you can always finetune it to get better performance out of it.
Remarkably, the prediction of weights works for previously unseen model architectures too. Design your own model, run its architecture by the GNN to predict the trained weights, and voilà, you have a model that is now ready for use -- and you can always finetune it to get better performance out of it.
Very cool!
--
[a] https://arxiv.org/pdf/2110.13100.pdf -- Abstract: "Deep learning has been successful in automating the design of features in machine learning pipelines. However, the algorithms optimizing neural network parameters remain largely hand-designed and computationally inefficient. We study if we can use deep learning to directly predict these parameters by exploiting the past knowledge of training other networks. We introduce a large-scale dataset of diverse computational graphs of neural architectures – DEEPNETS-1M– and use it to explore parameter prediction on CIFAR-10 and ImageNet. By leveraging advances in graph neural networks, we propose a hypernetwork that can predict performant parameters in a single forward pass taking a fraction of a second, even on a CPU. The proposed model achieves surprisingly good performance on unseen and diverse networks. For example, it is able to predict all 24 million parameters of a ResNet-50 achieving a 60% accuracy on CIFAR-10. On ImageNet, top-5 accuracy of some of our networks approaches 50%. Our task along with the model and results can potentially lead to a new, more computationally efficient paradigm of training networks. Our model also learns a strong representation of neural architectures enabling their analysis."
It's kind of like a more advanced version of hyper-parameter tuning which has been done in more primitive forms for a long time. But this approach definitely seems powerful.
It's for instantly training/pretraining any architecture for one or more predefined tasks (e.g., you could have a GNN for visual tasks, a GNN for NLP tasks, etc.), but you could also have a multi-domain GNN which accepts two inputs: an architecture and a domain.
Research use: Can we figure out how and why these GNNs can predict trained weights for previously unseen architectures without actually having to train the models? Can we simplify the GNNs? Can we use the knowledge gained from studying these GNNs to speed-up conventional training?
Aha. Using this as an accelerator for Neural Architecture Search would indeed be very neat.
I suppose there’s some value in possibly finding radically different networks (though I wonder if it’d fare well in such outlier regions). Perhaps if/since the GNN model is also differentiable you could invert it to maximize some compute/quality trade off point.
With this you could train a GNN on the various models you tested to speed-up training new ones.
As far as I can tell, SOTA ("state of the art") is 99% on CIFAR-10 [1]. The first entry in wikipedia, for 2010, shows 21.1% error rate, so 78.9% accuracy. Which is to say a 60% accuracy rate is far from useful to put it mildly.
Essentially, with that low an accuracy rate, I don't think there's reason to think the predictions are meaningful in a practical sense. But maybe there's something I'm missing in accuracy description.
[1] https://en.wikipedia.org/wiki/CIFAR-10
But I find that figure impressive: It implies that I can concoct a newfangled architecture for visual recognition, put it through one of these GNNs, and begin training at ~50% top-5 accuracy. Also, AFAIK this is the first effort of its kind; I would expect the figures to improve over time, as usual.
Granted the comparison is 2500 iterations of sgd which isn't a lot. I haven't played with cifar in a while, but that's probably not a ton of savings.
In essense this detects those building blocks and plugs the known weights in for you.
But reducing the training cost would lower the barrier to access.
what could possibly go wrong? ah, the exit cond.....
All in all I'm just incredibly excited to see how this line of research plays out and desperately want to get involved in it myself. I was actually considering learning JAX just the other day specifically so I could look more into the topic, but I'm lazy so I never got around to it.
This has been bugging me for a while. I can't articulate it well but I feel there's a hole in the logic. What exactly is the "program finder" finding? A better version of the program at finding... what exactly? You need a stop condition for the recursion.
I've been struggling to find the right terms to Google to find out more about the field, but it looks like "hypernetwork" was what I was looking forward. I highly, highly doubt that the first AGI is going to be designed by hand by humans -- I'd be willing to bet money that the first AGI is going to be recursively designed by multiple layers of constructor AIs just like what you're proposing.
Incredibly exciting idea IMO, and I'm confused why there doesn't appear to be more interest in it.
Regular AI isn't even done by hand anymore. We just pick data and learning parameters. At the current state we'd be picking higher and higher categories for learning. At the lowest level we pick data about cats to write something that recognizes cats. At a higher level we pick data about things that recognize things and so on... and so on.
How does extremely high level training data effect the low level accuracy of the AI at the lowest level? I would presume that it would effect it negatively as technically all AI is estimation. So estimations on estimations are less accurate.
https://en.wikipedia.org/wiki/Godel_machine
I don't believe AGI is going to conform to our rigid, symmetrical, boxy human-made architectures with their nice, neat flow patterns and powers-of-two factors everywhere -- those architectures work great for what they're designed to do, but achieving sentience/true intelligence just really doesn't feel like it's going to come out of a feedforward MLP, or even a transformer (in my opinion).
The most complicated and capable neural networks we know of are the organic, messy, squishy, yet organized electric meat lumps in our own skulls, produced by billions of years of being exposed to the same loss function over and over and over, but with unlimited ways to adapt in order to minimize that loss function. Nature makes awesome, fractally, insanely complex networks that we could never design by hand. I think it's only sensible to assume that AGI will work similarly.
Of course, this is all just armchair speculation. I'm not an expert, just an enthusiastic data scientist who thinks these next twenty years in AI are going to be a heck of a freaking ride.
But, I suppose arriving at such a formal definition doesn't really require that the thing being described be particularly amenable to our understanding.
However, I'm not sure that "if we ran it, if it is safe, it could prove itself to be safe" is sufficient to address the concerns.
If it isn't safe, then running at all may spell doom, so, "if it is safe, it could demonstrate that after we turn it on" doesn't really address that, because it doesn't give us any assurance before we turn it on.
Now, if we started with something with substantially sub-human overall intelligence, and which wasn't really agent-y (so that, if it was a little unsafe, it would at least not be catastrophically unsafe), but which was more equipped to formally prove things about itself and potential modifications of itself than humans are equipped to formally prove things about it, then we could task that thing with formally proving safety properties about itself, and of also proving safety properties about its successor, and do that before running its successor, and iterate this process to produce increasingly intelligent programs and perhaps also eventually agentic ones, while always having a safety proof of each before we run it..
But, I'm not really sure how plausible this route is? Like, even assuming we do reach AGI, safe or unsafe, I'm not sure this is a plausible route of getting there safely.
WANN paper: https://arxiv.org/pdf/1906.04358.pdf
Just kidding: After skimming the article, this looks like necessary progress. I myself (as a layman) had assumed this is already how they did it - although I'm sure there's a lot more to it than that.
——-
If you wanted to find best architecture in order to maximize accuracy, why not just train a model to predict accuracy (not parameters) given architecture and then optimize over the model?
This seems similar to optimizing any expensive black box function. Fit a cheap approximation (i.e., surrogate model) and then optimize over cheap model.