The reason speculative decoding shows diminishing returns in batched workloads is because the principle of both is the same. Speculative decoding predicts a group of tokens and verifies this group using the main model…
>LLMs are still bad at low-level hardware optimizations, but really good at high-level composition. I disagree. While yes they don't have all the architectural quirks of every GPU memorized, they are able to extract…
For me writing manpages as a spec worked well. While the format is unintuitive and tedious (at least to me), I used AI to format the manpages from my descriptions and rendered to PDF to review. Though it's been mainly…
The GPU has INT4, INT8, BF16 and FP16. Notably no FP8 or FP4.The official GPTQ-Int4 release from Qwen is a great quant for this but custom kernels are still rare for this hardware.
I'm not arguing it's right. Frankly, I think it's stupid the way things are. But I can understand why some people make such choice. I guess my argument mainly is that people who take their luggage are not stupid,…
In general the airlines just ask you to leave your luggage. If they were legally obliged to replace all your items, they would inform you of such. On international flights, an airline is liable for up to $1700 per the…
Sarcasm aside, while still not acceptable, some people might not have the means to buy new items to replace what they lost in a crash. So it is understandable for some people to make the choice of taking their luggage…
[deleted]
Which of course is different than the market price of an asset. You can't say sell something at intrinsic value if the market thinks differently.
>Genuine question, in what scenario is the self hosting setup and maintenance worth it? Maybe if you're a huge org with a dedicated security team and so on, which could easily handle managing such service. I guess at a…
>If you can't distinguish bad from good with LLMs, you might as well be throwing crap at the wall hoping it will stick. This is why I think LLMs are more of a tool for the expert rather than for the novice. They give…
What value? What kind of logistics value could they possible provide if they don't have logistics in the first place? It of course is problem of the consumer, as consumer is the one returning the item and the one who…
I think you have misunderstood what dropshipping means. In dropshipping the business won't have any foreign made inventory and neither will they have any in their country. Instead they will hook you up with a Chinese…
#1 is the website ships globally with no restrictions. A lot of Chinese businesses ship to any country. A local business with their own stock generally won't handle shipping to every single country around the globe.…
That's a dropshipping site for AliExpress products shipping from China. You'll save money if you buy the product straight from AliExpress without a middleman in between. However as you said, the toothbrush doesn't…
Yeah sorry I might've mixed up 8051 and 8086. Not totally sure, main point was that it's an old ass chip design used in modern products. These chips with 70s design are very common in ordinary "non computing" products,…
Yeah I totally agree if we consider Intel's innovation in the CPU space or lack thereof in the 21st century. I have seen more innovation happen in the CPU space in the last 5 years than the 20 years prior to that. This…
You have to look more far back than just the Pentium 4 days. Intel absolutely were some of the greatest innovators on the industry if you look at 70s, 80s and 90s. I would say they simply were the best, but dropped the…
>but the transcripts the AI makes of me are so hilariously My experience on AI transcripts is different: I use auto (AI) generated captions on YouTube for every video and while there sure are some mistakes with…
Yes, that was my point exactly. Due to nice weather, ice cream consumption and drowning begin to correlate and it's hard to draw conclusions from this correlation alone. As an example for coffee consumption it could…
Ah, that explains, thanks! I was scratching my head for a while wondering if I am missing the point totally... seems like I did
And ice cream consumption is correlated with drowning. Many times correlation just doesn't mean that much by itself.
At least in the South of EU there are places with zero checks in place. I mean there are organizations operating in the Mediterranean whose whole purpose is to offer transport for people outside the EU without any…
More discussion: https://news.ycombinator.com/item?id=41040445
Now I don't know what the parent said, but if you mean frightening as in taking up people's time, worry not, as commonly in small orgs such meetings don't take place in reality even though the minutes might indicate so.…
The reason speculative decoding shows diminishing returns in batched workloads is because the principle of both is the same. Speculative decoding predicts a group of tokens and verifies this group using the main model…
>LLMs are still bad at low-level hardware optimizations, but really good at high-level composition. I disagree. While yes they don't have all the architectural quirks of every GPU memorized, they are able to extract…
For me writing manpages as a spec worked well. While the format is unintuitive and tedious (at least to me), I used AI to format the manpages from my descriptions and rendered to PDF to review. Though it's been mainly…
The GPU has INT4, INT8, BF16 and FP16. Notably no FP8 or FP4.The official GPTQ-Int4 release from Qwen is a great quant for this but custom kernels are still rare for this hardware.
I'm not arguing it's right. Frankly, I think it's stupid the way things are. But I can understand why some people make such choice. I guess my argument mainly is that people who take their luggage are not stupid,…
In general the airlines just ask you to leave your luggage. If they were legally obliged to replace all your items, they would inform you of such. On international flights, an airline is liable for up to $1700 per the…
Sarcasm aside, while still not acceptable, some people might not have the means to buy new items to replace what they lost in a crash. So it is understandable for some people to make the choice of taking their luggage…
[deleted]
Which of course is different than the market price of an asset. You can't say sell something at intrinsic value if the market thinks differently.
>Genuine question, in what scenario is the self hosting setup and maintenance worth it? Maybe if you're a huge org with a dedicated security team and so on, which could easily handle managing such service. I guess at a…
>If you can't distinguish bad from good with LLMs, you might as well be throwing crap at the wall hoping it will stick. This is why I think LLMs are more of a tool for the expert rather than for the novice. They give…
What value? What kind of logistics value could they possible provide if they don't have logistics in the first place? It of course is problem of the consumer, as consumer is the one returning the item and the one who…
I think you have misunderstood what dropshipping means. In dropshipping the business won't have any foreign made inventory and neither will they have any in their country. Instead they will hook you up with a Chinese…
#1 is the website ships globally with no restrictions. A lot of Chinese businesses ship to any country. A local business with their own stock generally won't handle shipping to every single country around the globe.…
That's a dropshipping site for AliExpress products shipping from China. You'll save money if you buy the product straight from AliExpress without a middleman in between. However as you said, the toothbrush doesn't…
Yeah sorry I might've mixed up 8051 and 8086. Not totally sure, main point was that it's an old ass chip design used in modern products. These chips with 70s design are very common in ordinary "non computing" products,…
Yeah I totally agree if we consider Intel's innovation in the CPU space or lack thereof in the 21st century. I have seen more innovation happen in the CPU space in the last 5 years than the 20 years prior to that. This…
You have to look more far back than just the Pentium 4 days. Intel absolutely were some of the greatest innovators on the industry if you look at 70s, 80s and 90s. I would say they simply were the best, but dropped the…
>but the transcripts the AI makes of me are so hilariously My experience on AI transcripts is different: I use auto (AI) generated captions on YouTube for every video and while there sure are some mistakes with…
Yes, that was my point exactly. Due to nice weather, ice cream consumption and drowning begin to correlate and it's hard to draw conclusions from this correlation alone. As an example for coffee consumption it could…
Ah, that explains, thanks! I was scratching my head for a while wondering if I am missing the point totally... seems like I did
And ice cream consumption is correlated with drowning. Many times correlation just doesn't mean that much by itself.
At least in the South of EU there are places with zero checks in place. I mean there are organizations operating in the Mediterranean whose whole purpose is to offer transport for people outside the EU without any…
More discussion: https://news.ycombinator.com/item?id=41040445
Now I don't know what the parent said, but if you mean frightening as in taking up people's time, worry not, as commonly in small orgs such meetings don't take place in reality even though the minutes might indicate so.…