> Together’s $20M seed funding to build open-source
This is a huge contrast. Stability AI’s bread and butter is proprietary, not open source. Together seem to have a track record of open sourcing both code and models.
To be fair, Stability AI have open sourced StableLM, which is welcome. But overall Stability exaggerate and lie about their openness.
It doesn't sound like it's trying to build a single model, it wants to build a network/community of them. One of which is their own. Probably sharing/coordinating training resources and other support systems? The things that are usually out of financial/technical scope for most smaller OSS AI projects. I could see smaller companies with open models spending to be a part of that or other financially backed OSS projects.
Maybe it's more hands-on than just a hosted service like huggingface? I don't really know the market, so just speculating.
Does someone here know concretely what the product will likely be? Is it a replicate/modal/banana competitor, an OpenAI/Anthropic competitor, a Stability AI competitor, or something else
I've noticed a big trend by commercial entities claiming to be "Open Source" but actually charging for licenses. Just because your code is in a public repo on github doesn't mean it's open source. If it's not an open source license it's not open.
Open source is overrated, in my opinion. I’d rather a healthy commercial entity driving development and supporting it, with visible source that I can modify as a big plus.
The whole OSI open source thing is nice, but it’s not going to matter to me much either way. I’m more worried that the open source project is no longer maintained or unsupported or development is slow and it gets obsolete. Your values, and thus your opinion, may be different.
Since you mention it in connection to Together/RedPajama, are you saying that what they are publishing isn't actually open source? Most of the stuff I see them publish is licensed under Apache-2.0 which is a FOSS license, so I find it weird to mention it here.
Didn't OpenAI promise the same thing when it first came out? I'm not fully invested in the space yet...so if that was completely different than RedPajama I apologize.
I would also check out their 3B model. I tested it on launch with LoRA fine-tuning and found it to be surprisingly capable despite its size. I think a lot of people are skipping past testing it because it only has 3B params.
> to empower innovation and creativity by providing leading open-source generative AI models and a cloud platform that makes AI accessible to anyone, anywhere.
Since it's generally hard to earn money on the first part, I'm guessing they expect to make money on the latter part, assuming "anyone, anywhere" meaning people that can do business with US entities + people who can afford their service.
No mention of their alignment strategy, if any. I feel like alignment is even more important in open models since it’ll be much easier to train a “but evil” version, right?
23 comments
[ 2.7 ms ] story [ 65.0 ms ] threadThis is a huge contrast. Stability AI’s bread and butter is proprietary, not open source. Together seem to have a track record of open sourcing both code and models.
To be fair, Stability AI have open sourced StableLM, which is welcome. But overall Stability exaggerate and lie about their openness.
Maybe it's more hands-on than just a hosted service like huggingface? I don't really know the market, so just speculating.
Does someone here know concretely what the product will likely be? Is it a replicate/modal/banana competitor, an OpenAI/Anthropic competitor, a Stability AI competitor, or something else
This is especially true of AI projects.
The whole OSI open source thing is nice, but it’s not going to matter to me much either way. I’m more worried that the open source project is no longer maintained or unsupported or development is slow and it gets obsolete. Your values, and thus your opinion, may be different.
E.g. the whole tantrum Amazon had over the Elastic Search relicensing or Google briefly banning SSPL from their code hosting platform.
He's well known for producing open source.
It's the most realistic open source attempt at recreating the Facebook LLaMA LLM, from scratch, in a way that supports commercial usage.
They released their full 2.6TB training dataset last month, and it's significant: https://simonwillison.net/2023/Apr/17/redpajama-data/
They've also started releasing new, commercially-usable openly licensed LLM models trained on that data. You can try one of those out here: https://huggingface.co/togethercomputer/RedPajama-INCITE-Ins...
Edit: https://huggingface.co/togethercomputer/RedPajama-INCITE-Bas...
> to empower innovation and creativity by providing leading open-source generative AI models and a cloud platform that makes AI accessible to anyone, anywhere.
Since it's generally hard to earn money on the first part, I'm guessing they expect to make money on the latter part, assuming "anyone, anywhere" meaning people that can do business with US entities + people who can afford their service.
IMO, this sounds more like an excuse to cash out the $20M check