Electricity (on continental US) is pretty cheap assuming you already have the hardware: Running at a full load of 1000W for every second of the year, for a model that produces 100 tps at 16 cents per kWh, is $1200 USD.…
Not wrong, but I think it's more accurate to say: Mamba is an architecture for the middle layers of the network (the trunk) which assumes decoding takes place through an autoregressive sequence (popping out tokens in…
This is just standard Fourier theory of being able to apply dense global convolutions with pointwise operations in frequency space? There’s no mystery here. It’s no different than a more general learnable…
They also didn't have nuclear weapons to use in global conflicts over resources.
I've always found curriculum learning incredibly hard to tune and calibrate reliably (even more so than many other RL approaches!). Reward scales and horizon lengths may vary across tasks with different difficulty,…
Thief of Time by Terry Prachett has a great minor bit about characters who are naming themselves after colors running out of human made labels, as they have to get increasingly esoteric with the names. It's fun to see…
It's from the depth of the computation, not the work
In the middle term, I almost feel less productive using modern GPT-5/Claude Sonnet 4 for software dev than prior models, precisely because they are more hands off and less supervised. Because they generate so much code,…
Useful and interesting but likely still dangerous in production without connecting to formal verification tools. I know o3 is far from state of the art these days but it's great at finding relevant literature and…
For people who are interested Kokkos (a C++ library for writing portable kernels) also has a naming scheme for hierarchical parallelism. They use ThreadTeam, Thread (for individual threads within a group), and…
I can't reproduce this, possibly it's another environment error or the problem has been fixed in the versions I am using. (uv 0.7.11, zsh 5.9, python 3.13.5, MacOS 15.3.1)
It does? What problems have you had?
Yep! Optimize (solve in infinite dimensions) and then discretize onto a finite basis has typically led to much better and stable methods than a discretize and then optimize approach. Time-scale calculus is a pretty…
Does anyone know why they added minibatch advantage normalization (or when it can be useful)? The paper they cite "What matters in on-policy RL" claims it does not lead to much difference on their suite of test…
I am not sure why HN has mostly LANL posts. Otherwise though it is a combination of things. Machine learning applications for NATSec & fundamental research have become more important (see FASST, proposed last year), the…
All the problems of reproducibility in Python notebooks (https://arxiv.org/abs/2308.07333, https://leomurta.github.io/papers/pimentel2019a.pdf) with the power of a terminal.
How many people still play centaur chess?
ah, yes, the spooky similarities of hilbert spaces and probability theory
They likely mean on any of the current era of NISQ-like devices (https://en.wikipedia.org/wiki/Noisy_intermediate-scale_quant...) like this one or quantum annealers.
Yep, this is also the problem that self-driving cars have with the "accidents per mile" metric.
I love this paper, but something I think is often missed when it comes up is that you CAN hear the shape of many drums if you restrict the shape space, for example with a prior of "what a drum should look like" Zelditch…
This problem seems like it would be very similar to the Low-Rank + Sparse decompositions that used to be popular in audio-visual filtering.
These were two separate and mostly unrelated PEPs but otherwise that's correct.
This is neat and links to some great papers. I wish the comparison was with OpenMP tasks though; I’ve heard Rayon has a reputation for being a bit slow
Typically no, although BLAS software engineers occasionally write HPC strassen's implementations and papers about them. In my opinion there's a few reasons why they're not common in BLAS libraries: - They're a bit less…
Electricity (on continental US) is pretty cheap assuming you already have the hardware: Running at a full load of 1000W for every second of the year, for a model that produces 100 tps at 16 cents per kWh, is $1200 USD.…
Not wrong, but I think it's more accurate to say: Mamba is an architecture for the middle layers of the network (the trunk) which assumes decoding takes place through an autoregressive sequence (popping out tokens in…
This is just standard Fourier theory of being able to apply dense global convolutions with pointwise operations in frequency space? There’s no mystery here. It’s no different than a more general learnable…
They also didn't have nuclear weapons to use in global conflicts over resources.
I've always found curriculum learning incredibly hard to tune and calibrate reliably (even more so than many other RL approaches!). Reward scales and horizon lengths may vary across tasks with different difficulty,…
Thief of Time by Terry Prachett has a great minor bit about characters who are naming themselves after colors running out of human made labels, as they have to get increasingly esoteric with the names. It's fun to see…
It's from the depth of the computation, not the work
In the middle term, I almost feel less productive using modern GPT-5/Claude Sonnet 4 for software dev than prior models, precisely because they are more hands off and less supervised. Because they generate so much code,…
Useful and interesting but likely still dangerous in production without connecting to formal verification tools. I know o3 is far from state of the art these days but it's great at finding relevant literature and…
For people who are interested Kokkos (a C++ library for writing portable kernels) also has a naming scheme for hierarchical parallelism. They use ThreadTeam, Thread (for individual threads within a group), and…
I can't reproduce this, possibly it's another environment error or the problem has been fixed in the versions I am using. (uv 0.7.11, zsh 5.9, python 3.13.5, MacOS 15.3.1)
It does? What problems have you had?
Yep! Optimize (solve in infinite dimensions) and then discretize onto a finite basis has typically led to much better and stable methods than a discretize and then optimize approach. Time-scale calculus is a pretty…
Does anyone know why they added minibatch advantage normalization (or when it can be useful)? The paper they cite "What matters in on-policy RL" claims it does not lead to much difference on their suite of test…
I am not sure why HN has mostly LANL posts. Otherwise though it is a combination of things. Machine learning applications for NATSec & fundamental research have become more important (see FASST, proposed last year), the…
All the problems of reproducibility in Python notebooks (https://arxiv.org/abs/2308.07333, https://leomurta.github.io/papers/pimentel2019a.pdf) with the power of a terminal.
How many people still play centaur chess?
ah, yes, the spooky similarities of hilbert spaces and probability theory
They likely mean on any of the current era of NISQ-like devices (https://en.wikipedia.org/wiki/Noisy_intermediate-scale_quant...) like this one or quantum annealers.
Yep, this is also the problem that self-driving cars have with the "accidents per mile" metric.
I love this paper, but something I think is often missed when it comes up is that you CAN hear the shape of many drums if you restrict the shape space, for example with a prior of "what a drum should look like" Zelditch…
This problem seems like it would be very similar to the Low-Rank + Sparse decompositions that used to be popular in audio-visual filtering.
These were two separate and mostly unrelated PEPs but otherwise that's correct.
This is neat and links to some great papers. I wish the comparison was with OpenMP tasks though; I’ve heard Rayon has a reputation for being a bit slow
Typically no, although BLAS software engineers occasionally write HPC strassen's implementations and papers about them. In my opinion there's a few reasons why they're not common in BLAS libraries: - They're a bit less…