HN Explorer

Ask HN: Assume GTP-X arrives in 6 months and is 100X – what now?

3 points by dflock ↗ HN
After reading https://www.lesswrong.com/posts/N6vZEnCn6A95Xn39p/are-we-in-an-ai-overhang I was thinking...

What would a 100x improvement on GTP-3 do to your employability, for example? What changes would this bring in the short term? Medium term? What should one do to stay afloat in the medium term?

For the purposes of this question, I don't care about the details of the network/GTP etc... just the outcomes of a 100x improvement in results.

5 comments

[ 0.29 ms ] story [ 19.4 ms ] thread
minimaxir
GPT-3 is not a 100x improvement over GPT-2.
TomMarius
Are you sure? When I look at the GPT-2 examples, and remember my experience with it when you could try on talktotransformer.com, I would say it is.

GPT-2 would start to ramble incomprehensibly almost every time. It would forget context after a few words. It would get into loops. To make something resembling a conversation you had to heavily influence it.

minimaxir
It's better, just not 100x better, which is a big difference in practice.

Quantitatively about 5x of the output is more useful in my usage, which is not enough to upend society.

dflock
Doesn't really matter, for the purposes of this question.
dflock
How far can you get in some jobs by _just_ being _really_ good at NLP - i.e. without any insight or domain knowledge/reasoning, beyond what's needed for the NLP?