Based on rumors, it seems that because the demand is so high and supply so short, that Nvidia is having to select who gets the cards. I bet they're likely thinking about who can do the most impactful work and OpenAI…
For sure, I didn't mean to imply OpenAI's time-to-market was the main reason for my singularity comment. Say what you want about OpenAI, and trust me I think there's a lot of material there, but you can't say they don't…
So nutty. I didn't think they were going to drop such a huge bomb on the industry. We're firmly in "singularity" territory where the pace of acceleration is so fast, huge strides are being made every day / week, it…
Interesting paper. My confidence on the following idea isn't high, but I have an idea I don't think a lot of people agree with me on. The authors pretty convincingly show that transformers don't generalize well. I…
I bet you're right. Even if you take into account that a data center is a monster consumer of energy, in the grand scheme of things it's not that big. Some back of the envelope math: Global electrical production in 2022…
What's funny is that even though the DGX GH200 is some of the most powerful hardware available, there's such a voracious demand that it's not gonna be enough to quench it. In fact, this is one of those cases where I…
> You found one of your ideas appears in a recently published paper. You can no longer work on it. This is one of the things I thought of right away when ChatGPT got released last year. "God, there's probably so many…
I just realized that Justine was the person responsible for the massive reduction in the memory footprint of the Llama models back in March.[1] Super impressive! These are my favorite kinds of blog posts. [1]…
For sure. Just to be clear, I'm not saying the situation we're in where we have to release it to the general public is a great situation to be in. But I think we're at a point where there's not any optimal solutions,…
I have to disagree. Not releasing it to the public makes it more dangerous. One major downside of all development being done in private is that the AI can very easily be co-opted by our self-appointed betters and you…
Now someone build a bot that goes around looking for rogue bots and warns sysadmins.
I'm sure I'm not the only one, but I'd love to see a write-up on that. Prod your colleague to put it somewhere on YT, HN, Reddit or somewhere.
I agree with you on that. If it's less than 1% then it does make more sense to think of it that way.
Really? I've always been the opposite for some reason. I find it easier to visualize 20% of a pie chart, or of a crowd, or a length, etc. It's a lot harder for me to think about 1/5 of a crowd or 1/5 of a length. It's…
We should all be so lucky. It's always sad when it happens though. RIP
I'm curious as to what your take on all this recent progress is Gwern. I checked your site to see if you had written something, but didn't see anything recent other than your very good essay "It Looks Like You’re Trying…
Q: How many psychiatrists does it take to change a lightbulb? A: None, the lightbulb has to want to change.
I feel like I always learn something new whenever I read Gwern's articles. I found this idea particularly hilarious for some reason: ceiling-mounted infrared beam emitters+guidance infrared camera, steered by segmenting…
My first thought was that a built-in GPU could measure certain parameters about the display and adjust the color profile to be "pro" level accurate. Most default / "auto" profiles for monitors and TVs are accurate…
Thanks for posting the link, because I was about to ask. It seems like printing out the reference on a home printer would make the accuracy dependent on whether or not your printer was well calibrated (and then you're…
Based on rumors, it seems that because the demand is so high and supply so short, that Nvidia is having to select who gets the cards. I bet they're likely thinking about who can do the most impactful work and OpenAI…
For sure, I didn't mean to imply OpenAI's time-to-market was the main reason for my singularity comment. Say what you want about OpenAI, and trust me I think there's a lot of material there, but you can't say they don't…
So nutty. I didn't think they were going to drop such a huge bomb on the industry. We're firmly in "singularity" territory where the pace of acceleration is so fast, huge strides are being made every day / week, it…
Interesting paper. My confidence on the following idea isn't high, but I have an idea I don't think a lot of people agree with me on. The authors pretty convincingly show that transformers don't generalize well. I…
I bet you're right. Even if you take into account that a data center is a monster consumer of energy, in the grand scheme of things it's not that big. Some back of the envelope math: Global electrical production in 2022…
What's funny is that even though the DGX GH200 is some of the most powerful hardware available, there's such a voracious demand that it's not gonna be enough to quench it. In fact, this is one of those cases where I…
> You found one of your ideas appears in a recently published paper. You can no longer work on it. This is one of the things I thought of right away when ChatGPT got released last year. "God, there's probably so many…
I just realized that Justine was the person responsible for the massive reduction in the memory footprint of the Llama models back in March.[1] Super impressive! These are my favorite kinds of blog posts. [1]…
For sure. Just to be clear, I'm not saying the situation we're in where we have to release it to the general public is a great situation to be in. But I think we're at a point where there's not any optimal solutions,…
I have to disagree. Not releasing it to the public makes it more dangerous. One major downside of all development being done in private is that the AI can very easily be co-opted by our self-appointed betters and you…
Now someone build a bot that goes around looking for rogue bots and warns sysadmins.
I'm sure I'm not the only one, but I'd love to see a write-up on that. Prod your colleague to put it somewhere on YT, HN, Reddit or somewhere.
I agree with you on that. If it's less than 1% then it does make more sense to think of it that way.
Really? I've always been the opposite for some reason. I find it easier to visualize 20% of a pie chart, or of a crowd, or a length, etc. It's a lot harder for me to think about 1/5 of a crowd or 1/5 of a length. It's…
We should all be so lucky. It's always sad when it happens though. RIP
I'm curious as to what your take on all this recent progress is Gwern. I checked your site to see if you had written something, but didn't see anything recent other than your very good essay "It Looks Like You’re Trying…
Q: How many psychiatrists does it take to change a lightbulb? A: None, the lightbulb has to want to change.
I feel like I always learn something new whenever I read Gwern's articles. I found this idea particularly hilarious for some reason: ceiling-mounted infrared beam emitters+guidance infrared camera, steered by segmenting…
My first thought was that a built-in GPU could measure certain parameters about the display and adjust the color profile to be "pro" level accurate. Most default / "auto" profiles for monitors and TVs are accurate…
Thanks for posting the link, because I was about to ask. It seems like printing out the reference on a home printer would make the accuracy dependent on whether or not your printer was well calibrated (and then you're…