Yeah that was disapointing given the title. ChatGPT didn't spring full-formed out of nowhere. It's progress upon an existing body of work that is totally opaque to the uninitiated but a reasonable facsimile can be approximated out if you spend enough time on the right discords and http://huggingfaces.co and Google Colab. ML algos and their results shouldn't be limited to an Ivory tower of Arvix papers.
I tried them all, and continue trying when new things come out. No open model is capable of even remotely approaching the capabilities of GPT-3 or ChatGPT. Even if you had an equivalent trained model, running it would require enormous hardware.
While Stable Diffusion is pretty comparable to DALL-E 2 even on small hardware (such as M1 MacBook Air), this doesn't hold true for large language models. These need much, much more VRAM.
It's most likely because of the actual runtime size of the model. All the open models are sized for consumer-grade devices, and thus 10x-100x smaller than whatever OpenAI runs (probably around 100+ GB in VRAM, maybe even some multiples more). This is one of the main reasons why their API makes business sense - it's not practical at all to run models like GPT-3 yourself, and training them costs incredible amount of money too.
You can run GPT-J and GPT-Neo on (2) 24Gb GPUs, like a pair of 3090s. Not to mention Stable Diffusion which has much lower requirements to produce amazing AI images. Now, if you want to train a huge model from scratch that’s a different story, but running your own server is definitely possible and improving every day.
How much would 2 24gb GPUs cost? Unfortunately I think it’s still unaffordable for most
Maybe you could rent GPU space on a service like vast.ai. But I imagine it would be difficult to set up, and at that point currently you’re better off just using GPT3…
FYI I am planning to add a straightforward way to deploy dialog agents backed by OpenAIs new model text-davinci-003 to my site aidev.codes over the new few days if I have time.
So you will be able to mainly focus on the prompt template and the rest will be handled including the hosting, probably by something like a template app built in the system I have now. The main thing missing to enable easy creation of dialog agents is for me to add a completions endpoint and maybe things like socket.io integration in the template app.
I'm 90% sure this was written by / with assistance from ChatGPT; I previously prompted it for some React examples and some of this structure of how to setup the env is extremely familiar.
23 comments
[ 3.9 ms ] story [ 58.0 ms ] threadI wonder if one of the open models would work the same
While Stable Diffusion is pretty comparable to DALL-E 2 even on small hardware (such as M1 MacBook Air), this doesn't hold true for large language models. These need much, much more VRAM.
ChatGPT runs on a heavily modified version of GPT3 nicknamed “GPT3.5” that is also not available via API.
Is there a working demo, video, screenshots, or git repo for this solution that gives us a glimpse of what to expect once this is built?
Not something you can run on your own machine (or data center) unfortunately.
Maybe you could rent GPU space on a service like vast.ai. But I imagine it would be difficult to set up, and at that point currently you’re better off just using GPT3…
https://github.com/kingoflolz/mesh-transformer-jax/#gpt-j-6b
They even have a google colab notebook thingie. Not much setup needed, just an account.
https://colab.research.google.com/github/kingoflolz/mesh-tra...
"Building a chat app with GPT-3, ReactJS, and NextJS..."
Not
"Roll out your own ChatGPT"
which is rather unrealistic.
On first skim, it looks like a useful guide so good luck with it.
So you will be able to mainly focus on the prompt template and the rest will be handled including the hosting, probably by something like a template app built in the system I have now. The main thing missing to enable easy creation of dialog agents is for me to add a completions endpoint and maybe things like socket.io integration in the template app.
https://stackoverflow.com/questions/74924661/typeerror-opena...