There is an optimal chess program and there is an optimal AI algorithm. The former is merely practically uncomputable, the latter is theoretically uncomputable.
I think it’s possible AI job disruption (or the threat thereof) could lead to a new labor movement capable of uniting people across the isle. It remains to be seen, but I think if the threat of a “permanent underclass”…
At least for me, I think half the value in building to learn was that the knowledge and skills acquired in the process, especially cursory skills and knowledge, might be useful in the future, even if there was no…
On-prem AI is another solution. I’m surprised more companies aren’t leaning in this direction due to security/IP concerns. My employer is planning to spend several million on local AI hardware.
In the field of pure mathematics this might be true, but it has implications regardless for applied math, engineering, and physics.
I agree it is tautological in some sense. It is more precise to say compression requires prediction, and rather than sampling (as in a generative model), we use the probability distribution to efficiently encode…
I’m not exactly sure what distinction you are trying to make, but a compressor that generalizes from currently observed day to potential future data _is_ a better compressor, as it will have better predictions as it…
This is a lot less surprising when you learn how non-LZ compressors work, that is, by modeling a probability distribution and using those probabilities to encode information in the minimum number of bits required to…
Really cool project. I wonder what the future of LLM inference will look like. The Talaas demo is promising, but using an ASIC with weights in ROM means you can’t update the model (weights or architecture) without…
Once you have a mask of the positions you want to compress, you can generate a shuffle index vector from that mask to place the desired elements in the low part of the vector. You can expand the mask into nibble-sized…
I think this is one approach to AI safety and interpretability that could work, but would require labs to slow down to figure out how to extract circuits/algorithms out of trained LLMs rather than deploying the opaque…
https://abseil.io/fast/hints.html
I’m starting to think the probability of RSI within 12 months is more like 99% I’m not sure it will be FOOM, maybe it will require AIs to iterate on hardware to get orders of magnitude more compute/storage/energy which…
RTFM
Even worse, people will discuss/diagnose/troubleshoot with their LLM of choice, some of which will be valuable, and the only way to get access to said information will be through the LLM provider’s paywall. I think we…
Perhaps, but I think what people mean by intelligence is something that learns and adapts. If LLMs couldn’t do in context learning I don’t think people would think of them as AI, more as a kind of queryable database via…
Yeah I don’t understand this port at all other than as a big marketing stunt. It would have made far more sense, for reliability, efficiency, cost, etc., every metric really, to use or write a source to source…
Alienware does the same thing, and as far as I could figure out, there is no way to stop it. The enshitification of Windows at this point is incredible.
Agreed on Sedra and Smith vs Razavi. Razavi uploaded lectures to YouTube that helped me a ton at the time.
One interesting feature of signed vs unsigned in languages where signed overflow is undefined is that signed numbers _behave_ like mathematical integers, whereas unsigned integers (because overflow is valid behavior)…
I personally use a 3x6 Corne for non-gaming tasks. I would recommend it for anyone that uses a keyboard for more than a few hours a day to preemptively reduce the risk of RSI, especially programmers. The small layout…
Indeed. Their single most important piece of evidence not providing any strong indication the decline is from LLMs does not bode well for their argument. If it were the case you would expect the slope to become more…
I largely agree. Split, ortho/columnar, and tenting is 90% of the strain benefits. The biggest benefit from layout change (for programmers at least) is moving symbols and numbers to a layer where they share positions…
There are momentum indices. Momentum is actually a strange phenomenon from the perspective of the efficient market hypothesis because it does not have an obvious risk to balance its premium compared to other empirical…
[flagged]
There is an optimal chess program and there is an optimal AI algorithm. The former is merely practically uncomputable, the latter is theoretically uncomputable.
I think it’s possible AI job disruption (or the threat thereof) could lead to a new labor movement capable of uniting people across the isle. It remains to be seen, but I think if the threat of a “permanent underclass”…
At least for me, I think half the value in building to learn was that the knowledge and skills acquired in the process, especially cursory skills and knowledge, might be useful in the future, even if there was no…
On-prem AI is another solution. I’m surprised more companies aren’t leaning in this direction due to security/IP concerns. My employer is planning to spend several million on local AI hardware.
In the field of pure mathematics this might be true, but it has implications regardless for applied math, engineering, and physics.
I agree it is tautological in some sense. It is more precise to say compression requires prediction, and rather than sampling (as in a generative model), we use the probability distribution to efficiently encode…
I’m not exactly sure what distinction you are trying to make, but a compressor that generalizes from currently observed day to potential future data _is_ a better compressor, as it will have better predictions as it…
This is a lot less surprising when you learn how non-LZ compressors work, that is, by modeling a probability distribution and using those probabilities to encode information in the minimum number of bits required to…
Really cool project. I wonder what the future of LLM inference will look like. The Talaas demo is promising, but using an ASIC with weights in ROM means you can’t update the model (weights or architecture) without…
Once you have a mask of the positions you want to compress, you can generate a shuffle index vector from that mask to place the desired elements in the low part of the vector. You can expand the mask into nibble-sized…
I think this is one approach to AI safety and interpretability that could work, but would require labs to slow down to figure out how to extract circuits/algorithms out of trained LLMs rather than deploying the opaque…
https://abseil.io/fast/hints.html
I’m starting to think the probability of RSI within 12 months is more like 99% I’m not sure it will be FOOM, maybe it will require AIs to iterate on hardware to get orders of magnitude more compute/storage/energy which…
RTFM
Even worse, people will discuss/diagnose/troubleshoot with their LLM of choice, some of which will be valuable, and the only way to get access to said information will be through the LLM provider’s paywall. I think we…
Perhaps, but I think what people mean by intelligence is something that learns and adapts. If LLMs couldn’t do in context learning I don’t think people would think of them as AI, more as a kind of queryable database via…
Yeah I don’t understand this port at all other than as a big marketing stunt. It would have made far more sense, for reliability, efficiency, cost, etc., every metric really, to use or write a source to source…
Alienware does the same thing, and as far as I could figure out, there is no way to stop it. The enshitification of Windows at this point is incredible.
Agreed on Sedra and Smith vs Razavi. Razavi uploaded lectures to YouTube that helped me a ton at the time.
One interesting feature of signed vs unsigned in languages where signed overflow is undefined is that signed numbers _behave_ like mathematical integers, whereas unsigned integers (because overflow is valid behavior)…
I personally use a 3x6 Corne for non-gaming tasks. I would recommend it for anyone that uses a keyboard for more than a few hours a day to preemptively reduce the risk of RSI, especially programmers. The small layout…
Indeed. Their single most important piece of evidence not providing any strong indication the decline is from LLMs does not bode well for their argument. If it were the case you would expect the slope to become more…
I largely agree. Split, ortho/columnar, and tenting is 90% of the strain benefits. The biggest benefit from layout change (for programmers at least) is moving symbols and numbers to a layer where they share positions…
There are momentum indices. Momentum is actually a strange phenomenon from the perspective of the efficient market hypothesis because it does not have an obvious risk to balance its premium compared to other empirical…
[flagged]