This is simply wrong. Backprop has the same asymptotic time complexity as forward.
Which material ongoing issues are we ignoring? The paper is mainly talking about how the mundane problems we're already starting to have could lead to an irrecoverable catastrophe, even without any sudden betrayal or…
One of the authors here. I don't think we anthropomorphize AI as some sort of God. Here's a more prosaic analogy that might be helpful. Imagine tomorrow there's a new country full of billions of extremely conscientious,…
> as we notice these things, we pass laws against it Well, the claim is that that's the sort of thing that will get harder once humans aren't involved in most important decisions. > which is we created all this stuff,…
> we have categorized the ways it goes rogue (monopoly, extortion, etc) and responded adequately. This objection is a reasonable one. But the point of the paper is that a lot of the ways we have of addressing these…
> A society whose tagline is "alive and free in some senses" is already dystopian! Haha. Well we might agree about that - that description covers a wide range of possibilities. If you have ideas about what a plausible…
I appreciate your engagement, and I don't really have a plan myself to address these problems, but I don't really know what to do with "Socialism" as a recommendation. Care to elaborate what you think I, or anyone…
These are all good points about our use of language, thanks for the feedback. Maybe "disempowerment" is a bit of a red herring, or a misleading problem to focus on. The reason we didn't spend more time on clarifying…
Yep, a major missing piece in this entire problem / discussion is how to characterize how much "power" "humans" have had. My best idea so far is to characterize the sorts of outcomes that can be feasibly steered towards…
> What incentives do any humans have to so totally delegate the functioning of the core levers of societal power that they're unable to prevent their own extinction? Because it'll be more effective at every step than…
Last author here. I agree that states already have little incentive to effectively represent their citizens. But they could have even less! What would it look like to face these issues but more directly? Ending…
In many senses, yes. But the empowered ones still needed to keep most of the rest of the people happy and healthy enough to work, most of the time. That's what we're saying will change. In fact, it'll be worse: Humans…
Last author here. Good point, I agree that the move to an entirely self-sustaining machine economy would require extra time, and that would drag out the time to extinction even the worst case scenario by this mechanism.…
Update: the code is here: https://github.com/jerryqhyu/distill_bayes_net
I agree that priors over aspects of the world would be more useful, but I don't think that they're important in making natural intelligence powerful. In my experience, the important thing is to make your prior really…
BNNs certainly have their uses, but I think people in general found that it's a better use of compute to fit a larger model on more data than to try to squeeze more juice from a given small dataset + model. Usually…
Thanks for pointing that out!
This approach characterizes a different type of uncertainty than BNNs do, and the approaches can be combined. The BNN tracks uncertainty about parameters in the NN, and mixture density nets track the noise distribution…
I still am excited by Dex (https://github.com/google-research/dex-lang/) and still write code in it! I have a bunch of demos and fixes written, and am just waiting for Dougal to finish his latest re-write before I can…
I think we used a distill.pub template. Also Jerry wrote some custom BNN fitting code in javascript. I'll ask my co-authors to open-source it.
Sure, instead of saying "choose" a prior, you could say "elicit". But I think in this context, focusing on a practitioner's prior knowledge is missing the point. For the sorts of problems we use NNs for, we don't…
I would argue against the idea that "MLE is just Bayes with a flat prior". The power of Bayes usually comes mainly from keeping around all the hypothesis that are compatible with the data, not from the prior. This is…
Good point. We wrote this pre-double descent, and a massively overparameterized model would make a nice addition to the tutorial as a baseline. However, if you want a rich predictive distribution, it might still make…
I agree that Bayesian neural networks haven't been worth it in practice for many applications, but I think the main problem is that it's usually better to spend your compute training a single set of weights for a larger…
I agree choosing priors is hard, but choosing ReLU versus LeakyReLU versus sigmoid seems like a problem with using neural nets in general, not Bayesian neural nets in particular. Am I misunderstanding?
This is simply wrong. Backprop has the same asymptotic time complexity as forward.
Which material ongoing issues are we ignoring? The paper is mainly talking about how the mundane problems we're already starting to have could lead to an irrecoverable catastrophe, even without any sudden betrayal or…
One of the authors here. I don't think we anthropomorphize AI as some sort of God. Here's a more prosaic analogy that might be helpful. Imagine tomorrow there's a new country full of billions of extremely conscientious,…
> as we notice these things, we pass laws against it Well, the claim is that that's the sort of thing that will get harder once humans aren't involved in most important decisions. > which is we created all this stuff,…
> we have categorized the ways it goes rogue (monopoly, extortion, etc) and responded adequately. This objection is a reasonable one. But the point of the paper is that a lot of the ways we have of addressing these…
> A society whose tagline is "alive and free in some senses" is already dystopian! Haha. Well we might agree about that - that description covers a wide range of possibilities. If you have ideas about what a plausible…
I appreciate your engagement, and I don't really have a plan myself to address these problems, but I don't really know what to do with "Socialism" as a recommendation. Care to elaborate what you think I, or anyone…
These are all good points about our use of language, thanks for the feedback. Maybe "disempowerment" is a bit of a red herring, or a misleading problem to focus on. The reason we didn't spend more time on clarifying…
Yep, a major missing piece in this entire problem / discussion is how to characterize how much "power" "humans" have had. My best idea so far is to characterize the sorts of outcomes that can be feasibly steered towards…
> What incentives do any humans have to so totally delegate the functioning of the core levers of societal power that they're unable to prevent their own extinction? Because it'll be more effective at every step than…
Last author here. I agree that states already have little incentive to effectively represent their citizens. But they could have even less! What would it look like to face these issues but more directly? Ending…
In many senses, yes. But the empowered ones still needed to keep most of the rest of the people happy and healthy enough to work, most of the time. That's what we're saying will change. In fact, it'll be worse: Humans…
Last author here. Good point, I agree that the move to an entirely self-sustaining machine economy would require extra time, and that would drag out the time to extinction even the worst case scenario by this mechanism.…
Update: the code is here: https://github.com/jerryqhyu/distill_bayes_net
I agree that priors over aspects of the world would be more useful, but I don't think that they're important in making natural intelligence powerful. In my experience, the important thing is to make your prior really…
BNNs certainly have their uses, but I think people in general found that it's a better use of compute to fit a larger model on more data than to try to squeeze more juice from a given small dataset + model. Usually…
Thanks for pointing that out!
This approach characterizes a different type of uncertainty than BNNs do, and the approaches can be combined. The BNN tracks uncertainty about parameters in the NN, and mixture density nets track the noise distribution…
I still am excited by Dex (https://github.com/google-research/dex-lang/) and still write code in it! I have a bunch of demos and fixes written, and am just waiting for Dougal to finish his latest re-write before I can…
I think we used a distill.pub template. Also Jerry wrote some custom BNN fitting code in javascript. I'll ask my co-authors to open-source it.
Sure, instead of saying "choose" a prior, you could say "elicit". But I think in this context, focusing on a practitioner's prior knowledge is missing the point. For the sorts of problems we use NNs for, we don't…
I would argue against the idea that "MLE is just Bayes with a flat prior". The power of Bayes usually comes mainly from keeping around all the hypothesis that are compatible with the data, not from the prior. This is…
Good point. We wrote this pre-double descent, and a massively overparameterized model would make a nice addition to the tutorial as a baseline. However, if you want a rich predictive distribution, it might still make…
I agree that Bayesian neural networks haven't been worth it in practice for many applications, but I think the main problem is that it's usually better to spend your compute training a single set of weights for a larger…
I agree choosing priors is hard, but choosing ReLU versus LeakyReLU versus sigmoid seems like a problem with using neural nets in general, not Bayesian neural nets in particular. Am I misunderstanding?