Coding and math are easily verifiable, language and prose is not. As a result, I don't think LLMs will reach this state easily.
>Huh? You simply don't turn on the TV when the kid is there. If you want to save even more money and energy, don't get a TV. If there is a TV at a restaurant and the kid sees it, do you tell them to turn it off for you?…
Again, having the TV on to watch the news in the morning is not the same thing as plopping the child in front of cocomelon and letting them brainrot in the morning.
I'm just not convinced watching the news in the morning on the TV with our kid in the other room eating breakfast is going to ruin my child. The way the post is framed, it makes it seem like you have to completely…
>Most people use password123 as password, and like using the same password on every site, so saying that's bad is a bit silly? That's a ridiculous false equivalence and you know it. Choosing a poor password is nowhere…
From the perspective of other people, most parents will almost always be considered a bad parent. Parenting is hard, and strangers are always brutal judges of you. Best not to concern yourself with the opinions of…
Do keep in mind that the vast majority of people posting in this thread of grown up with TVs in the background. The 0-3 year rule is a little silly. Like sure, of course, don't let the kid just mindlessly watch TV. But…
I agree with the framing but the claim is too strong. I don't think Grassmannian manifolds are necessarily a better representation than simple query/key inner products. It's a valid solution, and it does have advantages…
Clever, I wonder what would happen if you did this in reverse? Like, if we froze the weights then added some layers trained on a bunch of weird edge cases then tried to convert those weights back into branches. QA…
A byproduct of working remote is the amount of human connection in general has gone way down. The only time I "see" the people I work with anymore is when we're on a meeting. Meetings that could have been an email, in…
For those saying this creates a threat to Nvidia's deathgrip on the market, it really doesn't The main reason people don't use Huawei hardware is due to regulation. If you're in the US you'd straight up get your…
Yeah, there's a tradeoff. pre-training with QAT needs more data and a ton of hardware, so it's easier to just take an existing open weight model and quantize it. This works, but it'll underperform a model that had…
>I'm a little skeptical of going below 4-bit quants due to the potential for significant degradation in quality There's literature on this actually, if you want to get that low the model really should have quantization…
How else would they get their marketing stories unless the agents can "break out" of containment?
>You literally ask the agent to do something and it does it It's not that simple, you have to remember at the end of the day these things are just doing next token prediction. If you don't give it the proper tokens to…
If you want to do a 1-bit model you have to QAT at pre-training with way more data than chinchilla to compensate for the cliffs (like 50x). Quantization on an existing pre-trained model will almost always collapse at…
We use the ONNX runtime for small models in the browser https://github.com/microsoft/onnxruntime
just switched to doom emacs this month, absolutely loving it tbh
yeah, but I mean even prose specific claude-isms aside; the information itself is a weird patchwork of concepts
I notice a lot of these AI written articles share this pattern where they'll present idea 1, then idea 2, and finally idea 3 which is some amalgamation of idea 1 and 2. Claude especially will present hybrid options and…
Has anyone read this yet? I'm extremely skeptical of any book published after ~2024 just because of how much AI slop is in physical books these days. Looks interesting, but I'd like to see some reviews on the content…
I think a part of this is that people tend to undervalue their own skills and expertise when talking about these anecdotes. A lot of people in the comments do have a software engineering background. People at different…
I don't know if I agree with the premise that having access to AI results in dulling human intellect. I feel like to get to Terry's level you need a combination of passion and aptitude for the subject. People that don't…
The big take away for is the fact that the ONLY reason why chatgpt was able to get to this counterexample was because of the knowledge of the person driving the conversation. I don't think chatgpt could have come to…
Nothing wrong with vibecoding a toy project.
Coding and math are easily verifiable, language and prose is not. As a result, I don't think LLMs will reach this state easily.
>Huh? You simply don't turn on the TV when the kid is there. If you want to save even more money and energy, don't get a TV. If there is a TV at a restaurant and the kid sees it, do you tell them to turn it off for you?…
Again, having the TV on to watch the news in the morning is not the same thing as plopping the child in front of cocomelon and letting them brainrot in the morning.
I'm just not convinced watching the news in the morning on the TV with our kid in the other room eating breakfast is going to ruin my child. The way the post is framed, it makes it seem like you have to completely…
>Most people use password123 as password, and like using the same password on every site, so saying that's bad is a bit silly? That's a ridiculous false equivalence and you know it. Choosing a poor password is nowhere…
From the perspective of other people, most parents will almost always be considered a bad parent. Parenting is hard, and strangers are always brutal judges of you. Best not to concern yourself with the opinions of…
Do keep in mind that the vast majority of people posting in this thread of grown up with TVs in the background. The 0-3 year rule is a little silly. Like sure, of course, don't let the kid just mindlessly watch TV. But…
I agree with the framing but the claim is too strong. I don't think Grassmannian manifolds are necessarily a better representation than simple query/key inner products. It's a valid solution, and it does have advantages…
Clever, I wonder what would happen if you did this in reverse? Like, if we froze the weights then added some layers trained on a bunch of weird edge cases then tried to convert those weights back into branches. QA…
A byproduct of working remote is the amount of human connection in general has gone way down. The only time I "see" the people I work with anymore is when we're on a meeting. Meetings that could have been an email, in…
For those saying this creates a threat to Nvidia's deathgrip on the market, it really doesn't The main reason people don't use Huawei hardware is due to regulation. If you're in the US you'd straight up get your…
Yeah, there's a tradeoff. pre-training with QAT needs more data and a ton of hardware, so it's easier to just take an existing open weight model and quantize it. This works, but it'll underperform a model that had…
>I'm a little skeptical of going below 4-bit quants due to the potential for significant degradation in quality There's literature on this actually, if you want to get that low the model really should have quantization…
How else would they get their marketing stories unless the agents can "break out" of containment?
>You literally ask the agent to do something and it does it It's not that simple, you have to remember at the end of the day these things are just doing next token prediction. If you don't give it the proper tokens to…
If you want to do a 1-bit model you have to QAT at pre-training with way more data than chinchilla to compensate for the cliffs (like 50x). Quantization on an existing pre-trained model will almost always collapse at…
We use the ONNX runtime for small models in the browser https://github.com/microsoft/onnxruntime
just switched to doom emacs this month, absolutely loving it tbh
yeah, but I mean even prose specific claude-isms aside; the information itself is a weird patchwork of concepts
I notice a lot of these AI written articles share this pattern where they'll present idea 1, then idea 2, and finally idea 3 which is some amalgamation of idea 1 and 2. Claude especially will present hybrid options and…
Has anyone read this yet? I'm extremely skeptical of any book published after ~2024 just because of how much AI slop is in physical books these days. Looks interesting, but I'd like to see some reviews on the content…
I think a part of this is that people tend to undervalue their own skills and expertise when talking about these anecdotes. A lot of people in the comments do have a software engineering background. People at different…
I don't know if I agree with the premise that having access to AI results in dulling human intellect. I feel like to get to Terry's level you need a combination of passion and aptitude for the subject. People that don't…
The big take away for is the fact that the ONLY reason why chatgpt was able to get to this counterexample was because of the knowledge of the person driving the conversation. I don't think chatgpt could have come to…
Nothing wrong with vibecoding a toy project.