In honor of Weizenbaum's Centenary, I asked ChatGPT to program ELIZA
Today, Jan 8th, is the 100th centenary of Joseph Weizenbaum's birth, so I asked ChatGPT to write ELIZA in js.
Long dialog (~10 rounds) short, here's where it ended up:
''' // A list of ELIZA's responses const responses = { "I feel": [ "Why do you feel {0}?", "What do you think is causing you to feel {0}?", "Do you often feel {0}?", "How do you normally deal with feeling {0}?" ], "I want": [ "Why do you want {0}?", "What led you to want this?", "Do you often find yourself wanting {0}?", "How do you think getting what you want will make you feel?" ], "I need": [ "Why do you feel like you need {0}?", "What led you to feel like you need this?", "Do you often find yourself feeling like you need {0}?", "How do you think getting what you need will make you feel?" ] };
// A list of keywords const keywords = Object.keys(responses);
// A function that generates a response function generateResponse(input) { // Check if the input contains any of the keywords for (let i = 0; i < keywords.length; i++) { if (input.includes(keywords[i])) { // If the input contains a keyword, return a random response // that includes the remainder of the input, with personal // pronouns conjugated correctly const response = responses[keywords[i]][Math.floor(Math.random() * responses[keywords[i]].length)]; return response.replace("{0}", input.substring(keywords[i].length + 1)); } } }
// Test the function console.log(generateResponse("I feel sad")); console.log(generateResponse("I want a new car")); console.log(generateResponse("I need some help"));
'''
My conclusion: It's not yet up to being a 1970s 13 year old programmer....yet. :-)
---- (Apparently ''' ... ''' doesn't do code in HN. Sorry for the wrap-around formatting.)
58 comments
[ 3.6 ms ] story [ 44.7 ms ] threadI think a lot of the time we think of singularity as 'the point at which the rules break because evolution of some parameter is infinite" but I think really what this term means is the point at which current rules are incapable of predicting the evolution of the system, resulting in an apparent infinity. That infinity probably is not physically meaningful, but it might as well be from the point of view of an actor before the new rules are known.
When we evolved from ape-ish things capable of only the most primitive communication into things capable of transmitting complex information about our world across generations we hit a rate of technological growth that might as well have been a vertical line compared to the previous history of life on earth.
When single-cellular life managed to capture mitochondria and unlock incredible chemical energy reserves that was another similarly vertical line in the complexity of life.
I'd say our history is a series of singularities that only look like gradual progress from the point of view of our own pseudo-infinity. We are a step in the process of the universe understanding itself but we aren't the last step, it would be silly to think so, given what we know about past leaps in the entropy creation rate of life.
Personally I think the General in AGI is real-valued (or possibly vector-valued) and definitely not a mere Boolean yes-or-no, and therefore it's fine to say ChatGPT is an AGI because it has the generality of "text on most subjects in most languages".
Well… sort of. I'm not sure about the Intelligence part of the label though, because it's making up for being very inefficient with the training set by running on hardware that outpaces synapses to the degree to which marathon runners outpace continental drift. But this leads to the question "Does a submarine swim?" (I.e. does it matter?)
https://github.com/BenWheatley/Studies-of-AI/blob/main/codin...
https://youtu.be/XX2rbcrXblk
Also used it to update the style on my website. Its solution has one bug that shows up on Chrome but not Safari, and was otherwise better than I could've done Googling because I didn't know the search term.
Fundamentally this approach is flawed. Can one really talk about the intelligence without talking about the environment in which that intelligence manifests itself?
1. AI will never be able to do X because that requires real intelligence/creativity
2. AI does X to a (super)human level
3. X does not require real intelligence/creativity, it is just search/statistics/etc.
with increasingly difficult values of X can not possibly continue forever, right?
And yet, it seems to happen so reliably.
What's really going to get spooky is someday some ChatGPT descendant is going to write a better ChatGPT and where does that sequence end?
Technological singularity, hopefully!
And if it can, there's no guarantee ChatGPT3 would be able to write a better ChatGPT4, and so on.
Exponential phenomena tend to need the right environment to sustain, and, by their nature, quickly outgrow, disturb/ruin, or consume their environment.
Not that there couldn't be an AI explosion (as far as I know), but there's really no particular reason to think one is imminent -- especially since we haven't seen step 1 yet.
Come on...
Also from a planar perspective we hit the end of the road for silicon transistors. All new designs must be three dimensional to increase density and a millennia of Moore's law would imply that somehow your computational resources grow faster than the speed of light. I mean you physically cannot travel fast enough to acquire interstellar resources for usage in computation when you are doubling every two years. You are talking about millions of galaxies filled with data centers by the end of the millenia, maybe the entire universe, I haven't checked the numbers.
All I know is that even 3% growth over 2000 years gives illogical results like colonizing a dozen milky ways.
I don't think either of these assumptions is reasonable.
There's also this DeepMind paper (https://www.deepmind.com/publications/an-empirical-analysis-...) saying people need to focus on better training more than data. Of course, there's not much more freely available text available anyway.
Starting from the current state, you could make ChatGPT much better programmer simply via brute-force approach:
It's not pretty, it's not cheap and it's not super robust, but neither is the code written by majority of programmers. And I'm sure you could make it understand the code even better using some kind of evolutionary algorithms, by letting it play with interpreter.In fact probably you can use a compiler and a human supplied test suit to generate training data for the network. Let it do the looping with the compiler where its output is fed into the compiler and if any error happens you feed the compiler output to the network. Then if it manages to write something which compiles you run the test and feed back any errors it got into the network.
If it ever manages to find a solution you add it to the dataset used to train the next version.
ChatGPT is like power-assisting prompt-craft. Special case tricks and tunings make casuals rate ChatGPT assisted prompt results higher than GPT-3 alone. Some of this is discussed here:
ChatGPT -- ChatGPT is a sibling model to InstructGPT, which is trained to follow an instruction in a prompt and provide a detailed response.
See: https://openai.com/blog/chatgpt/
InstructGPT -- We’ve trained language models that are much better at following user intentions than GPT-3 while also making them more truthful and less toxic, using techniques developed through our alignment research. These InstructGPT models, which are trained with humans in the loop, are now deployed as the default language models on our API.
See: https://openai.com/blog/instruction-following/
But, if you carefully craft prompts and prime continuations, it seems GPT-3.5 (the same new davinci models) lets you color outside ChatGPT's lines to achieve arguably even better results once you have your own crafted prompt.
For pre-formatted code leave two spaces at the start of the line.
There's no special notation for code. Despite this being, you know Hacker News.[...]
> There's no special notation for code
You just explained the special notation for code, which is so important that it is HN’s only block environment, and then turn around and deny it exists?
One observation Bob Bryla made was, count the number of words that begin with 'N' and treat each as a logical-negation. With a very small dictionary of additional negation words (e.g. can't) and a whitelist (e.g. nice) you could tell the sense of a statement or question pretty accurately.
Nope, never, not, nada, na, nobody and on and on.
"Some of ELIZA's responses were so convincing that Weizenbaum and several others have anecdotes of users becoming emotionally attached to the program, occasionally forgetting that they were conversing with a computer.[3] Weizenbaum's own secretary reportedly asked Weizenbaum to leave the room so that she and ELIZA could have a real conversation. Weizenbaum was surprised by this, later writing: "I had not realized ... that extremely short exposures to a relatively simple computer program could induce powerful delusional thinking in quite normal people."
Wikipedia states that a version of ELIZA is still available in emacs under m-x doctor or something similar. Not being an emacs user, I haven't verified that, but I thought it was interesting.
I guess that too is also a truism - "People in the future will look at our current technology and laugh", but somehow this one makes me feel things. The idea that what we have now is like a blunt 2x4 compared to the surgeon's scalpel that we might have in 60 years. (Or might not - who knows!)
https://corecursive.com/eliza-with-jeff-shrager/
> Weizenbaum, for his part, turned away from his own project’s expanding implications. He objected to the idea that something as subtle, intimate, and human as therapy could be reduced to code. He began to argue that fields requiring human compassion and understanding shouldn’t be automated. And he also worried about the same future that Alan Turing had described — one where chatbots regularly fooled people into thinking they were human. – https://99percentinvisible.org/episode/the-eliza-effect/
Out of curiosity, I today asked it to program the ELIZA in MAD-SLIP, the original (and now completely dead and obscure) language that ELIZA was originally written in. My expectation was that it would have no idea what MAD-SLIP is, or maybe would barely have an idea, and in any case would have no chance of doing what I asked and that it would, very reasonably, balk.
Instead, to my shock it made up a completely bogus meaning for the acronym: "Matched Activated Decomposition by Syntactic and Lexical Interchange Program", and actually wrote an ELIZA in this non-existing language, which was in reality just Python code using the rexexp (re) package, and was perfectly fine as far as it went.
Even scarier than confabulating a non-exisistent programming language, it confabulated a citation for it, claiming that it is described in Weizenbaum's Computer Power and Human Reason, which is utterly false!
Sigh. I fear that ChatGPT is merely bringing Weizenbaum's worst fears into reality! A seemingly authoritative AI that simply makes s#!t up! I'm feeling a bit like taking up arms at the moment.