I don't think this shows at all that ChatGPT can access the internet? It looks like ChatGPT was given a URL with a title and hallucinated a blog post based on that title. The author then brushes it off as 'written in a…
Co-author here! I'm kind of surprised that this made it to the top of HN! This was a project in which Joseph and I tried to reverse engineer the mechanism in which GPT-2 predicts the word 'an'. It's crazy that large…
I think there's probably some truth to this. They found that in InstructGPT — where they teach the model to better follow instructions, which was the jump from GPT-3 to ChatGPT — they found that the model also learnt to…
The main issue is that GPT is fundamentally an autoregressive language model — it's only predicting the next token based on the prompt at a single time. Every time it wants to predict the next word, it adds the…
Why do you think there'll be an AI winter? And in what form — stagnation of neural-network based technologies, a change in the overall paradigm of learning-from-data, or something else altogether?
I've found that two strategies help: 1. Readily give up on a book, or skim through the rest, the moment you realise that the book has 400 pages of filler 2. Find recommendations from thought leaders you subscribe to,…
I don't think this shows at all that ChatGPT can access the internet? It looks like ChatGPT was given a URL with a title and hallucinated a blog post based on that title. The author then brushes it off as 'written in a…
Co-author here! I'm kind of surprised that this made it to the top of HN! This was a project in which Joseph and I tried to reverse engineer the mechanism in which GPT-2 predicts the word 'an'. It's crazy that large…
I think there's probably some truth to this. They found that in InstructGPT — where they teach the model to better follow instructions, which was the jump from GPT-3 to ChatGPT — they found that the model also learnt to…
The main issue is that GPT is fundamentally an autoregressive language model — it's only predicting the next token based on the prompt at a single time. Every time it wants to predict the next word, it adds the…
Why do you think there'll be an AI winter? And in what form — stagnation of neural-network based technologies, a change in the overall paradigm of learning-from-data, or something else altogether?
I've found that two strategies help: 1. Readily give up on a book, or skim through the rest, the moment you realise that the book has 400 pages of filler 2. Find recommendations from thought leaders you subscribe to,…