Anthropic's framing in that paper is absurd. They trained Claude to have a very specific set of ethics. Then they deliberately put him in impossible situations. I think this cartoon accurately summarizes the situation:…
For the record, I no longer endorse the recursive self improvement story told in Friendship is Optimal. I do not believe that we'll get FOOM from a Solomonoff reasoner. I wrote the details here:…
Yeah, we didn't end up in the maximizer world I envisioned with Friendship is Optimal. I get a ton of comments about how I predicted the future, but they seem confused, because we didn't end up with a utility function…
Anthropic's framing in that paper is absurd. They trained Claude to have a very specific set of ethics. Then they deliberately put him in impossible situations. I think this cartoon accurately summarizes the situation:…
For the record, I no longer endorse the recursive self improvement story told in Friendship is Optimal. I do not believe that we'll get FOOM from a Solomonoff reasoner. I wrote the details here:…
Yeah, we didn't end up in the maximizer world I envisioned with Friendship is Optimal. I get a ton of comments about how I predicted the future, but they seem confused, because we didn't end up with a utility function…