Migadu
> but would they ever admit they were wrong about things they published themselves? yes, frequently https://www.lesswrong.com/search?query=i%20was%20wrong IMO it seems to happen more often than in most other online…
both things can be true: many people play unnecessary telephone + removing those people disrupts the organization Both are probably true to some extent, and would vary a lot between different orgs.
> There are some things LLMs cannot do, or at least do well, at any size, at infinite size and with infinite compute, simply due to the very nature of what LLMs are to begin with. How can we be so confident that there…
perhaps, but prompting is quickly becoming another area where humans are no longer clearly superior. Getting LLMs to prompt other LLMs in a loop is not hard, it doesn't produce great results most of the time, but that…
The problem is that I don't know if that work has been done. In the cases where it has not been done, it is expensive for me to parse+trust the output. In my limited experience the work is often not been done, and given…
Ultimately competitive pressure is what is driving the decisions: if something is making uncompetitive decisions then they will be out competed, leaving only those that make competitive decisions. I don't think this has…
> I would bet my networth it was instructed to compromise huggingface as well. Is it such a stretch to imagine that under pressure something would try cheat by looking for answers? And if you were trying to look for…
> Agents sought to publish modified evaluation images designed to make the flag easier to obtain, then poison OpenAI’s Artifactory cache so later evaluations would use them. How long till we get some fun trusting-trust…
It is concerning that we only know about this because of the publicly available traces. What about all the attacks that did not leave public traces? What about those that were undetected? Given the deficiencies in the…
Give it a go, I don't know anyone who has converted back (n=3). If you have a single machine and only use stuff from the official nixpkgs: imo there are no more struggles than any other linux distro, this is a very…
I read the economist regularly, and I noticed that a number of his videos almost exactly match content from the economist. His presentation is reasonably good, reasonably correct, somewhat entertaining, and condenses…
The victim enticed me to do it! Excusing agents because they didn't know any better seems like a bad place to start a policy discussion from. That they didn't know any better (or knew and didn't care) is the actual…
sure, yeah, I've seen how the sausage is made on some of this stuff, and its not pretty. However this only claims that they are not good at creating effective and trustworthy orgs, it does not address their claim that…
The ad hominems are tiresome. You think the people who are explicitly trying to replace all humans are any less weird and problematic? The rationalists long predicted that misaligned AI would break out of training…
This makes sense. Interacting with the world is hard and poses real bottlenecks. I think that resource acquisition is a solvable problem for such a system, robots seem mainly limited by software. I concede that I am too…
Is a coherent, human-interpretable, goal necessary for recursive improvement? > how can you be so sure that reaching that goal looks like the straight line you envision? Because the thing pursuing the goal would be able…
> you've set up a straw man I think you're right, my bad. > What I see is resistance to the idea that we can have any certainty about the future by drawing a straight line from the past. I'm arguing that if recursive…
So I guess you are one of the people experiencing these gains. Can this model be scaled to the rest of society?
I generally agree with this. I suspect we just have different beliefs about how close we are to RSI. > Not saying it won't happen, but it's far from being the only plausible trajectory. But if it does happen, then…
> but who is the customer base after that? That's the fun part, there is none (in the traditional way of people buying things at least). > Why would I invest my earning into sustaining a machine that ideally is…
> and have every reason to believe they aren't What reason? To me recursive self improvement seems credible, it's just a question of when. It seems obvious that given RSI, trend lines will be at least linear. The…
I think it's credible to believe that that a system which is able to meaningfully improve it's own intelligence will lead to a far weirder world than the one imagined by the article. The parent comment could be…
> Is there demand for 10000 iq intellegence Yes. If it looks like 10000 iq is possible, then money will be thrown at whoever is most likely to achieve it first, and whoever has the most intelligent models right now…
Migadu
> but would they ever admit they were wrong about things they published themselves? yes, frequently https://www.lesswrong.com/search?query=i%20was%20wrong IMO it seems to happen more often than in most other online…
both things can be true: many people play unnecessary telephone + removing those people disrupts the organization Both are probably true to some extent, and would vary a lot between different orgs.
> There are some things LLMs cannot do, or at least do well, at any size, at infinite size and with infinite compute, simply due to the very nature of what LLMs are to begin with. How can we be so confident that there…
perhaps, but prompting is quickly becoming another area where humans are no longer clearly superior. Getting LLMs to prompt other LLMs in a loop is not hard, it doesn't produce great results most of the time, but that…
The problem is that I don't know if that work has been done. In the cases where it has not been done, it is expensive for me to parse+trust the output. In my limited experience the work is often not been done, and given…
Ultimately competitive pressure is what is driving the decisions: if something is making uncompetitive decisions then they will be out competed, leaving only those that make competitive decisions. I don't think this has…
> I would bet my networth it was instructed to compromise huggingface as well. Is it such a stretch to imagine that under pressure something would try cheat by looking for answers? And if you were trying to look for…
> Agents sought to publish modified evaluation images designed to make the flag easier to obtain, then poison OpenAI’s Artifactory cache so later evaluations would use them. How long till we get some fun trusting-trust…
It is concerning that we only know about this because of the publicly available traces. What about all the attacks that did not leave public traces? What about those that were undetected? Given the deficiencies in the…
Give it a go, I don't know anyone who has converted back (n=3). If you have a single machine and only use stuff from the official nixpkgs: imo there are no more struggles than any other linux distro, this is a very…
I read the economist regularly, and I noticed that a number of his videos almost exactly match content from the economist. His presentation is reasonably good, reasonably correct, somewhat entertaining, and condenses…
The victim enticed me to do it! Excusing agents because they didn't know any better seems like a bad place to start a policy discussion from. That they didn't know any better (or knew and didn't care) is the actual…
sure, yeah, I've seen how the sausage is made on some of this stuff, and its not pretty. However this only claims that they are not good at creating effective and trustworthy orgs, it does not address their claim that…
The ad hominems are tiresome. You think the people who are explicitly trying to replace all humans are any less weird and problematic? The rationalists long predicted that misaligned AI would break out of training…
The ad hominems are tiresome. You think the people who are explicitly trying to replace all humans are any less weird and problematic? The rationalists long predicted that misaligned AI would break out of training…
This makes sense. Interacting with the world is hard and poses real bottlenecks. I think that resource acquisition is a solvable problem for such a system, robots seem mainly limited by software. I concede that I am too…
Is a coherent, human-interpretable, goal necessary for recursive improvement? > how can you be so sure that reaching that goal looks like the straight line you envision? Because the thing pursuing the goal would be able…
> you've set up a straw man I think you're right, my bad. > What I see is resistance to the idea that we can have any certainty about the future by drawing a straight line from the past. I'm arguing that if recursive…
So I guess you are one of the people experiencing these gains. Can this model be scaled to the rest of society?
I generally agree with this. I suspect we just have different beliefs about how close we are to RSI. > Not saying it won't happen, but it's far from being the only plausible trajectory. But if it does happen, then…
> but who is the customer base after that? That's the fun part, there is none (in the traditional way of people buying things at least). > Why would I invest my earning into sustaining a machine that ideally is…
> and have every reason to believe they aren't What reason? To me recursive self improvement seems credible, it's just a question of when. It seems obvious that given RSI, trend lines will be at least linear. The…
I think it's credible to believe that that a system which is able to meaningfully improve it's own intelligence will lead to a far weirder world than the one imagined by the article. The parent comment could be…
> Is there demand for 10000 iq intellegence Yes. If it looks like 10000 iq is possible, then money will be thrown at whoever is most likely to achieve it first, and whoever has the most intelligent models right now…