Ask HN: Is September 2021 the new 'eternal September'?

3 points by MichaelMoser123 2y ago ↗ HN

Both GPT-3 and GPT-4 mark this date as their 'knowledge cutoff date'. Maybe this date is chosen on purpose, in order to avoid a negative feedback loop, where the learning process of their own LLM is influenced by the output of bots that are powered by their own engines?

A public reminder: September 1993 was the old 'eternal September', where the big collaborative message board of olden times died, due to a sudden influx of new users. See: https://en.wikipedia.org/wiki/Eternal_September

1 comment

[ 3.5 ms ] story [ 35.2 ms ] thread

ilaksh 2y ago ↗

Training involved collecting and preparing a massive dataset and they can't just add to it on the fly without restarting the training. Training the model took a long time. Once they had the initial trained model they needed to test it. Then they started a huge additional training effort with human feedback.

All of those things take time.