Plenty of people are already miserable. Might as well try if you are no?
What does __syncthreads() do here exactly? I'm new to CUDA, could get the overall idea of the FlashAttention paper but not the details.
I would guess, to make sure the classic noob mistake of typing `if x = 3:` instead of `if x == 3:` stays a syntax error
I had missed this message but have to say I love the expression "orthogonal nightmares", really captures it well :)
If this happens we will just get more things done with the same amount of compute (see: Blinn's law). The demand for GPUs does not really come from algorithmic compute requirements but from social expectation of…
Javascript's package management is so much better than Python's IMO...
Most papers already have code, and videos are very common.
I understand better what you mean now. Clearly you've thought about this topic more than I have, so I don't think I can contribute much to the conversation unfortunately. "It still has to deliver value in return for the…
I don't see your point, to be honest. This US government could easily afford to host Reddit. The reasons it does not are cultural, not economic.
That's a fair point about allowing competition. But frankly, it just sucks that Reddit choses to purposefully hurt its service. With social media, there's also a strong moating effect that is fundamentally…
I'm not the type to hate on capitalism but Reddit's management is truly capitalism gone wrong. Ruining a public service for the profits of a few. What a waste.
Thanks for the info. I didn't know about the TomeSD stuff, really interesting. Why do you think that AITemplate is so much faster?
I find you post intriguing. What would you say are the major janks with torch.compile, and what issues are addressed by TVM/AITemplate but not by torch.compile? EDIT: If I understand correctly these libraries target…
Not to dismiss the guts of these explorers, but we have not been to the Moon in 50 years because it costs a fortune to get to when it's not much more than a floating rock (I'm being hyperbolic but the point is that…
`with` blocks in Python become very ugly when nested, I'm glad it's a single line personally
To me it seems like disposing of the object being destructured is the natural solution. But I can see where others will disagree.
I agree. I feel like `using async x = ...` would read much more nicely and avoid the confusion.
That's strange, what's the motivation behind not allowing destructuring? Seems very prudish.
Imagine that resource inherits from a class with an existing method called dispose. What happens then?
Saccadic masking probably comes into play: https://en.wikipedia.org/wiki/Saccadic_masking (brains "drops frames" while the eyes move, so "popping" during eye movement probably won't be perceptible)
What kind of bagel?
I see your point. I just went over the abstract again and I totally agree.
While I agree that the claims are hyperbolic, I think you are approaching this paper from the point of view of someone who knows a lot about sorting. Because of this, its normal that the claims of these guys who…
Sure, but the whole point is to reduce this kind of search to RL, which is a very general framework. Their paper shows that such a generic approach can solve a very specific problem, and solve it well. But, their paper…
I feel like your take is overly cynical. The fact that humans can do the same thing by hand is not really the point. The contribution lies in the fact that their method derived this improvement *automatically*, which is…
Plenty of people are already miserable. Might as well try if you are no?
What does __syncthreads() do here exactly? I'm new to CUDA, could get the overall idea of the FlashAttention paper but not the details.
I would guess, to make sure the classic noob mistake of typing `if x = 3:` instead of `if x == 3:` stays a syntax error
I had missed this message but have to say I love the expression "orthogonal nightmares", really captures it well :)
If this happens we will just get more things done with the same amount of compute (see: Blinn's law). The demand for GPUs does not really come from algorithmic compute requirements but from social expectation of…
Javascript's package management is so much better than Python's IMO...
Most papers already have code, and videos are very common.
I understand better what you mean now. Clearly you've thought about this topic more than I have, so I don't think I can contribute much to the conversation unfortunately. "It still has to deliver value in return for the…
I don't see your point, to be honest. This US government could easily afford to host Reddit. The reasons it does not are cultural, not economic.
That's a fair point about allowing competition. But frankly, it just sucks that Reddit choses to purposefully hurt its service. With social media, there's also a strong moating effect that is fundamentally…
I'm not the type to hate on capitalism but Reddit's management is truly capitalism gone wrong. Ruining a public service for the profits of a few. What a waste.
Thanks for the info. I didn't know about the TomeSD stuff, really interesting. Why do you think that AITemplate is so much faster?
I find you post intriguing. What would you say are the major janks with torch.compile, and what issues are addressed by TVM/AITemplate but not by torch.compile? EDIT: If I understand correctly these libraries target…
Not to dismiss the guts of these explorers, but we have not been to the Moon in 50 years because it costs a fortune to get to when it's not much more than a floating rock (I'm being hyperbolic but the point is that…
`with` blocks in Python become very ugly when nested, I'm glad it's a single line personally
To me it seems like disposing of the object being destructured is the natural solution. But I can see where others will disagree.
I agree. I feel like `using async x = ...` would read much more nicely and avoid the confusion.
That's strange, what's the motivation behind not allowing destructuring? Seems very prudish.
Imagine that resource inherits from a class with an existing method called dispose. What happens then?
Saccadic masking probably comes into play: https://en.wikipedia.org/wiki/Saccadic_masking (brains "drops frames" while the eyes move, so "popping" during eye movement probably won't be perceptible)
What kind of bagel?
I see your point. I just went over the abstract again and I totally agree.
While I agree that the claims are hyperbolic, I think you are approaching this paper from the point of view of someone who knows a lot about sorting. Because of this, its normal that the claims of these guys who…
Sure, but the whole point is to reduce this kind of search to RL, which is a very general framework. Their paper shows that such a generic approach can solve a very specific problem, and solve it well. But, their paper…
I feel like your take is overly cynical. The fact that humans can do the same thing by hand is not really the point. The contribution lies in the fact that their method derived this improvement *automatically*, which is…