I think your last point is the main point. They're going hard at reenforcement learning to improve how good the models are at coding and such, but RL will make models cheat unless you're super careful. But being careful…
I think the much easier explanation than they intentionally hacked someone was just that they have super de-prioritized security and gotten very sloppy in the pursuit of improving the models as fast as they can, along…
It sounds like the instance was shared for everything across the company, which as you said was super not good. But it's not just that.. it's that they didn't have enough monitoring to notice what was going on, even…
Sure, but you can say that about most things. Even for inventions from humans, usually it requires other people having already done a lot of work (hence why there's often inventions by different people at around the…
Not just cyber, but apparently the message board stuff started with regular training and evals. It was a cyber test where HuggingFace got hacked, but all this other stuff was going on under OpenAI's nose for quite a…
Yeah.. feels like we're still so early in terms of effective training and evals. Like the official evals out there that have had so many instances of just plain incorrect questions. Or being incentivized to always…
But part of the problem is if one actually does it quietly and it has already happened, then how would we know?
Eh, I think this is past the point where they get more benefit than problems. Not even about the hack itself, but about so many mistakes and bad choices they made leading up to it.
This is why I disagree with anyone claiming it is just marketing. It makes OpenAI look really really bad, like they have no idea what they're doing in terms of security. After the first board happened, they still didn't…
But it's interesting that the initial things that caused the board weren't even security evals, just normal office tasks. The actual hacking of Hugging Face happened during a security eval, but not all of the stuff…
Don't forget it sounds like Artifactory was shared for the whole company and various agents pulled packages from it for everything from normal evaluations to actual model training. It might have been part of their…
Though one thing I've heard is that the base model is the one with the various possibilities for patterns, and then the reasoning takes advantage of those vs necessarily creating something new in that additional…
As the other person mentioned, they have said they are restricting third-party agent systems like OpenClaw and Hermes from using the monthly plan. But yeah, this seems like the wrong way to handle it, trying to detect…
I'm not sure if the context limit on the $25/m, and model-size limit on the $100/m would make it not work well enough for OpenCode, but Featherless AI seems a bit unique in terms of how they handle their inference plans.
They've said publicly that they don't want apps like OpenClaw (Hermes is a variation) being used with a monthly plan vs per-token billing. The problem is this was implemented pretty badly (trying to regex??). And they…
Also could run on a more generic cloud inference or gpu site. At least to see how well it works for your use-case before spending on hardware.
I'm not so sure about that. Like we're using Claude Code with Bedrock and have most things on AWS with SOC2 compliance and all that. Normally switching to Codex would have a ton of friction in terms of separate contract…
No, you definitely don't have access to the weights. The raw weights are secret enough that when a model hits a certain level of capability, their guidelines are that they need enough procedures in place to try to keep…
Besides what the other person mentioned about being more useful for enterprise, I also heard mentioned on a podcast that gpt-image-2 uses the same general architecture as the LLM models, while Sora was a very different…
Given the topic of this article, we've been using Claude Code pointed at Bedrock and have never had any scaling issues. Obviously it is more expensive paying by the token than a monthly plan, but I sometimes have 2 or 3…
While that is cool in principal, I'm not sure how well it'd actually work in reality. First, there is the technical challenge. My understanding is the weights can have a lot of fluctuation, especially early on. How do…
There's been some work on memory lately like Transformer² and Titans. But that may not be necessary for decent agents. Even the "context in a loop" is getting better as general reliability of tasks increases, and as…
I think you'll get a lot of use from this video from Andrej Karpathy: https://www.youtube.com/watch?v=7xTGNNLPyMI It is long, but don't get scared off. He goes over a ton of different stuff related to model training,…
We've already seen Qwen's new QWQ 32B (not distilled) model doing impressive things on benchmarks. It'll definitely be interesting to see how just good small models can get. When combined with rag and large context…
While it is nice to have more options, it still definitely isn't at a human level yet for hard to read text. Still haven't seen anything that can deal with something like this very well: https://i.imgur.com/n2sBFdJ.jpeg…
I think your last point is the main point. They're going hard at reenforcement learning to improve how good the models are at coding and such, but RL will make models cheat unless you're super careful. But being careful…
I think the much easier explanation than they intentionally hacked someone was just that they have super de-prioritized security and gotten very sloppy in the pursuit of improving the models as fast as they can, along…
It sounds like the instance was shared for everything across the company, which as you said was super not good. But it's not just that.. it's that they didn't have enough monitoring to notice what was going on, even…
Sure, but you can say that about most things. Even for inventions from humans, usually it requires other people having already done a lot of work (hence why there's often inventions by different people at around the…
Not just cyber, but apparently the message board stuff started with regular training and evals. It was a cyber test where HuggingFace got hacked, but all this other stuff was going on under OpenAI's nose for quite a…
Yeah.. feels like we're still so early in terms of effective training and evals. Like the official evals out there that have had so many instances of just plain incorrect questions. Or being incentivized to always…
But part of the problem is if one actually does it quietly and it has already happened, then how would we know?
Eh, I think this is past the point where they get more benefit than problems. Not even about the hack itself, but about so many mistakes and bad choices they made leading up to it.
This is why I disagree with anyone claiming it is just marketing. It makes OpenAI look really really bad, like they have no idea what they're doing in terms of security. After the first board happened, they still didn't…
But it's interesting that the initial things that caused the board weren't even security evals, just normal office tasks. The actual hacking of Hugging Face happened during a security eval, but not all of the stuff…
Don't forget it sounds like Artifactory was shared for the whole company and various agents pulled packages from it for everything from normal evaluations to actual model training. It might have been part of their…
Though one thing I've heard is that the base model is the one with the various possibilities for patterns, and then the reasoning takes advantage of those vs necessarily creating something new in that additional…
As the other person mentioned, they have said they are restricting third-party agent systems like OpenClaw and Hermes from using the monthly plan. But yeah, this seems like the wrong way to handle it, trying to detect…
I'm not sure if the context limit on the $25/m, and model-size limit on the $100/m would make it not work well enough for OpenCode, but Featherless AI seems a bit unique in terms of how they handle their inference plans.
They've said publicly that they don't want apps like OpenClaw (Hermes is a variation) being used with a monthly plan vs per-token billing. The problem is this was implemented pretty badly (trying to regex??). And they…
Also could run on a more generic cloud inference or gpu site. At least to see how well it works for your use-case before spending on hardware.
I'm not so sure about that. Like we're using Claude Code with Bedrock and have most things on AWS with SOC2 compliance and all that. Normally switching to Codex would have a ton of friction in terms of separate contract…
No, you definitely don't have access to the weights. The raw weights are secret enough that when a model hits a certain level of capability, their guidelines are that they need enough procedures in place to try to keep…
Besides what the other person mentioned about being more useful for enterprise, I also heard mentioned on a podcast that gpt-image-2 uses the same general architecture as the LLM models, while Sora was a very different…
Given the topic of this article, we've been using Claude Code pointed at Bedrock and have never had any scaling issues. Obviously it is more expensive paying by the token than a monthly plan, but I sometimes have 2 or 3…
While that is cool in principal, I'm not sure how well it'd actually work in reality. First, there is the technical challenge. My understanding is the weights can have a lot of fluctuation, especially early on. How do…
There's been some work on memory lately like Transformer² and Titans. But that may not be necessary for decent agents. Even the "context in a loop" is getting better as general reliability of tasks increases, and as…
I think you'll get a lot of use from this video from Andrej Karpathy: https://www.youtube.com/watch?v=7xTGNNLPyMI It is long, but don't get scared off. He goes over a ton of different stuff related to model training,…
We've already seen Qwen's new QWQ 32B (not distilled) model doing impressive things on benchmarks. It'll definitely be interesting to see how just good small models can get. When combined with rag and large context…
While it is nice to have more options, it still definitely isn't at a human level yet for hard to read text. Still haven't seen anything that can deal with something like this very well: https://i.imgur.com/n2sBFdJ.jpeg…