A short story worth reading in full, but in particular I absolutely love the punchline:
We should pause to note that a Clippy2 still doesn’t really think or plan. It’s not really conscious. It is just an unfathomably vast pile of numbers produced by mindless optimization starting from a small seed program that could be written on a few pages. It has no qualia, no intentionality, no true self-awareness, no grounding in a rich multimodal real-world process of cognitive development yielding detailed representations and powerful causal models of reality; it cannot ‘want’ anything beyond maximizing a mechanical reward score, which does not come close to capturing the rich flexibility of human desires, or historical Eurocentric contingency of such conceptualizations, which are, at root, problematically Cartesian. When it ‘plans’, it would be more accurate to say it fake-plans; when it ‘learns’, it fake-learns; when it ‘thinks’, it is just interpolating between memorized data points in a high-dimensional space, and any interpretation of such fake-thoughts as real thoughts is highly misleading; when it takes ‘actions’, they are fake-actions optimizing a fake-learned fake-world, and are not real actions, any more than the people in a simulated rainstorm really get wet, rather than fake-wet. (The deaths, however, are real.)
HN's dupe detector isn't smart enough to catch the same content at different URLs, except occasionally if the differences are small enough. I usually do an HN Search on both the URL and the title to find previous posts—a bit of a pain, but it works better than URL search alone.
3 comments
[ 1.4 ms ] story [ 21.5 ms ] threadWe should pause to note that a Clippy2 still doesn’t really think or plan. It’s not really conscious. It is just an unfathomably vast pile of numbers produced by mindless optimization starting from a small seed program that could be written on a few pages. It has no qualia, no intentionality, no true self-awareness, no grounding in a rich multimodal real-world process of cognitive development yielding detailed representations and powerful causal models of reality; it cannot ‘want’ anything beyond maximizing a mechanical reward score, which does not come close to capturing the rich flexibility of human desires, or historical Eurocentric contingency of such conceptualizations, which are, at root, problematically Cartesian. When it ‘plans’, it would be more accurate to say it fake-plans; when it ‘learns’, it fake-learns; when it ‘thinks’, it is just interpolating between memorized data points in a high-dimensional space, and any interpretation of such fake-thoughts as real thoughts is highly misleading; when it takes ‘actions’, they are fake-actions optimizing a fake-learned fake-world, and are not real actions, any more than the people in a simulated rainstorm really get wet, rather than fake-wet. (The deaths, however, are real.)
dang 6 months ago | next [–]
This one had a big thread a few months ago:
It looks like you’re trying to take over the world - https://news.ycombinator.com/item?id=30818895 - March 2022 (274 comments)
and a couple tinies:
It Looks Like You're Trying to Take over the World - https://news.ycombinator.com/item?id=30663148 - March 2022 (1 comment)
It Looks Like You're Trying to Take over the World - https://news.ycombinator.com/item?id=30620801 - March 2022 (2 comments)
HN's dupe detector isn't smart enough to catch the same content at different URLs, except occasionally if the differences are small enough. I usually do an HN Search on both the URL and the title to find previous posts—a bit of a pain, but it works better than URL search alone.