HN Explorer

15 comments

[ 5.1 ms ] story [ 41.1 ms ] thread
aliljet
I think this is a clever way for OpenAI to trade access to GPT-4 and simultaneously decrease their own intellectual burden for their for-profit models. Here's the offending parts of the README.md:

"For a limited time, we will be granting GPT-4 access to those who contribute high quality evals. Please follow the instructions mentioned above and note that spam or low quality submissions will be ignored!"

nabakin
I'm surprised they would give it out when millions, if not billions are potentially at stake and following the LLaMA leak
Centigonal
Access to the inference API, not model code or weights.
nabakin
Are you sure? API access is already given to everyone I think

Edit: We have access through ChatGPT+, not the API. Got confused there.

not-chatgpt
to be fair, with alpaca, access to API is even dangerous
astrange
There wasn't a leak. You don't need to make it sound like we're in a cyberpunk RPG.
nabakin
what are you talking about? The LLaMA weights leaked from someone on 4chan.
astrange
Facebook released them with essentially zero effort to stop anyone from resharing them. I think they let anyone with a .edu email get them. It wasn't hacked or anything.
Tiberium
But they still didn't publish it into the _public_, like a public download link on their GitHub repo, so it's still a leak.
nabakin
Can we not have semantic arguments on HN? You know what I meant.
smoldesu
Facebook is still directly hosting the model, if they wanted to stop people from downloading it then it wouldn't be on their public CDN.
piqi
> Evals is a framework for evaluating OpenAI models and an open-source registry of benchmarks.

Is the purpose to know which of the models OpenAI offers is most suitable for your workload/app? Could I use this to know if the cheaper model is sufficient for a particular use-case?

nprateem
See, they are an open source company after all. They want you to help make their stuff better.