Interesting that this is the approach advocated in the leaked Google memo.
One thing is that it has received a lot of attention. For example the llama.cop project has significantly made it easier to build and run on consumer hardware. In addition, there are a lot of fine tunings available for that model.
This is a common strategy - when you don't know what the product is, rather than wait until your competitor gets a strong lead- undercut them by releasing your own product in the "spirit" of giving something away. Stir up interest and hope the dust cloud is enough to obscure your competitor long enough until you figure out how to monetize it.
Yup, and also remember to scoop up any interesting ideas generated and produced by people working on your "open" thing, then monetize that. Harnessing crowdsourced free work...
Guess no. That is the trick right? Let's see what people build. If it is good copycat it. If you build something very cool that makes money, be ready being sued by meta...
People built an entire ecosystem around llama. This could benefit FB, however their models are too limited to compete with chatGpt, so the open source tools arent particularly useful to them. FB is probably using better models by now. OTOH the open source community has an entire set of tools for the moment when Llama will be replaced with something better and completly open sourced model. at the moment , OSS seems to have benefited from their release.
Vicuna looks pretty good. But as said, commercial use not possible.
Why do you think that Llama can be replaced? I mean it is extremely costly to train that thing. And it is there even a clean open source data set for the task?
PS: wouldn't be surprised if Meta, OpenAi, or google will train something for a Billion $ in costs of compute or more.
the 2048 sequence length makes it uncompetitive, especially now that we are entering ~infinite-length bots. $1M for training is peanuts for FB. There are open datasets like redPajama and alternative models are (hopefully) coming up
The big deal here is releasing the model pre-trained. The training part would cost $1M in compute costs if you had to do it in-house.
The idea is that a company or developer downloads the model already trained on natural language. Then you add your additional documents and data and prompt it for answers internally, rather than through a service like OpenAI.
The model provided comes with enough legal strings attached that it's only really usable in the unofficial turn a blind eye to licenses context. i.e. Hobbyist, researches and casual use.
18 comments
[ 4.9 ms ] story [ 66.5 ms ] threadOne thing is that it has received a lot of attention. For example the llama.cop project has significantly made it easier to build and run on consumer hardware. In addition, there are a lot of fine tunings available for that model.
PS: Not an expert just speculating.
Why do you think that Llama can be replaced? I mean it is extremely costly to train that thing. And it is there even a clean open source data set for the task?
PS: wouldn't be surprised if Meta, OpenAi, or google will train something for a Billion $ in costs of compute or more.
The idea is that a company or developer downloads the model already trained on natural language. Then you add your additional documents and data and prompt it for answers internally, rather than through a service like OpenAI.
PS I wish I could get the NYT to write marketing pieces for my stuff!
More like handed out good replicas.
The model provided comes with enough legal strings attached that it's only really usable in the unofficial turn a blind eye to licenses context. i.e. Hobbyist, researches and casual use.