Neat. Very similar to tree-based speculation as they point out, and they also point how to combine them. Speculative decoding: Sample a linear output (next n tokens) from draft model, submit it to a verifier model. At…
> Almost every team that I’ve been talking to that is training a LLM right now talks about how they’re training a Chinchilla optimal model, which is remarkable given that basically everything in the LLM space changes…
Call me an optimist, but I think prompt injection just isn't as fundamental a problem as it seems. Having a single, flat text input sequence with everything in-band isn't fundamental to transformer: The architecture…
"They" as in OpenAI, when they trained the tokenizer, just dumped a big set of text data into a BPE (byte pair encoding) tokenizer training script, and it saw that string in the data so many times that it ended up…
> Models like ChatGPT aren’t eligible for the Hutter Prize for a variety of reasons, one of which is that they don’t reconstruct the original text precisely—i.e., they don’t perform lossless compression. Small nit: The…
> I'm glad that you enjoyed my previous responses, but I want to clarify that I was not pretending to be a Linux terminal. People who like to pooh-pooh generative AI systems as unable to be "truly creative" or to have…
Stories like this might as well be titled "Local change in Earth's magnetic field causes objects to float in the air". They shouldn't be read and then disputed because their details don't match the facts, they should be…
With the defaults of per_replica_batch=1, seq=2048 and gen_len=512, a completion takes about 20 seconds. I'm not sure yet what settings I'll end up with if I decide to play with this more. per_replica_batch=3, seq=1024,…
I'm running it comfortably on my 3090, although it's a really snug fit for the VRAM, and that's with a number of fixes to significantly reduce its memory use from https://github.com/AeroScripts/mesh-transformer-jax .
I used to think that this was only ever going to be possible with some sort of incomprehensible type trickery that I would never be able to understand. Then TypeScript came along and showed me that no, actually, in a…
The Internet often makes me feel old, but seeing Photopia called a "newish" game certainly put some spryness in my joints again. It's not that it's not a great game - it is - but even in its medium, I do think it should…
Aw. Well, it was worth a shot, that kind of mild homonym abuse is kind of direction that these sorts of titles usually tend to take. Fortunately the real story is more interesting.
My guess is it's either about a picture of Donald Knuth, or Knuth's reward checks, being put in picture frames.
"Nitpickers will complain that this isn’t the same as pickle.load(), since you have to pass a Class<T> token to choose what type of thing you want ahead of time. However, nothing is stopping you from passing…
Could be it just came across the right pair of eyes. It's quite reminiscent of the old EvilIx problem in Haskell, where you can break memory safety without doing anything explicitly marked unsafe:…
This turned out to be quite an angry post, so a quick preface: All of this is a knee-jerk response to a single sentence in your post. So, you know, please take with a grain of salt, not personally, etc.. I have to…
I don't know. Maybe if you've been reading bad science fiction with immortal geniuses running around everywhere, it might seem that living to 120 and having an IQ of 130 is leaving a lot on the table, but when you…
Just to make sure I'm not completely confused here: "Information can't be lost" and "you can't arrive to the same state of the universe through two different paths" are two ways to state exactly the same thing, right?…
I'll admit to not being a physicist and to being easily confused by the subject, so I'll just use a silly little metaphor to check whether I'm thinking straight, and try to avoid terminology I don't fully understand…
Only locally, though. The next galaxy cluster over has a uniquely privileged inertial reference frame, too, but it's in motion (and accelerating!) relative to ours.
AIT applies to: - Votes with at least three options. A simple majority vote between two options does in fact satisfy all of Arrow's fairness criteria (if only vacuously). - Ranked voting systems, i.e. ones where voters…
Just for the sake of having another solution for the same problem, here's the basically-zero-thought-required brute-force search solution in C: https://gist.github.com/arirahikkala/ef1a07dbd5a8ca3a077c03c... Not as fast…
In particular: The unit of GDP is in fact dollars/year (even if people rarely bother reporting the precise unit), whereas market cap is just dollars. They're as incomparable as velocity and position.
You did misunderstand. Read the next sentence: > The purpose of this frame is to allow an endpoint to gracefully stop accepting new streams (perhaps for a reboot or maintenance), while still finishing processing of…
> Fewer words, create a more powerful message. Fewer commas, create a sentence that makes more sense.
Neat. Very similar to tree-based speculation as they point out, and they also point how to combine them. Speculative decoding: Sample a linear output (next n tokens) from draft model, submit it to a verifier model. At…
> Almost every team that I’ve been talking to that is training a LLM right now talks about how they’re training a Chinchilla optimal model, which is remarkable given that basically everything in the LLM space changes…
Call me an optimist, but I think prompt injection just isn't as fundamental a problem as it seems. Having a single, flat text input sequence with everything in-band isn't fundamental to transformer: The architecture…
"They" as in OpenAI, when they trained the tokenizer, just dumped a big set of text data into a BPE (byte pair encoding) tokenizer training script, and it saw that string in the data so many times that it ended up…
> Models like ChatGPT aren’t eligible for the Hutter Prize for a variety of reasons, one of which is that they don’t reconstruct the original text precisely—i.e., they don’t perform lossless compression. Small nit: The…
> I'm glad that you enjoyed my previous responses, but I want to clarify that I was not pretending to be a Linux terminal. People who like to pooh-pooh generative AI systems as unable to be "truly creative" or to have…
Stories like this might as well be titled "Local change in Earth's magnetic field causes objects to float in the air". They shouldn't be read and then disputed because their details don't match the facts, they should be…
With the defaults of per_replica_batch=1, seq=2048 and gen_len=512, a completion takes about 20 seconds. I'm not sure yet what settings I'll end up with if I decide to play with this more. per_replica_batch=3, seq=1024,…
I'm running it comfortably on my 3090, although it's a really snug fit for the VRAM, and that's with a number of fixes to significantly reduce its memory use from https://github.com/AeroScripts/mesh-transformer-jax .
I used to think that this was only ever going to be possible with some sort of incomprehensible type trickery that I would never be able to understand. Then TypeScript came along and showed me that no, actually, in a…
The Internet often makes me feel old, but seeing Photopia called a "newish" game certainly put some spryness in my joints again. It's not that it's not a great game - it is - but even in its medium, I do think it should…
Aw. Well, it was worth a shot, that kind of mild homonym abuse is kind of direction that these sorts of titles usually tend to take. Fortunately the real story is more interesting.
My guess is it's either about a picture of Donald Knuth, or Knuth's reward checks, being put in picture frames.
"Nitpickers will complain that this isn’t the same as pickle.load(), since you have to pass a Class<T> token to choose what type of thing you want ahead of time. However, nothing is stopping you from passing…
Could be it just came across the right pair of eyes. It's quite reminiscent of the old EvilIx problem in Haskell, where you can break memory safety without doing anything explicitly marked unsafe:…
This turned out to be quite an angry post, so a quick preface: All of this is a knee-jerk response to a single sentence in your post. So, you know, please take with a grain of salt, not personally, etc.. I have to…
I don't know. Maybe if you've been reading bad science fiction with immortal geniuses running around everywhere, it might seem that living to 120 and having an IQ of 130 is leaving a lot on the table, but when you…
Just to make sure I'm not completely confused here: "Information can't be lost" and "you can't arrive to the same state of the universe through two different paths" are two ways to state exactly the same thing, right?…
I'll admit to not being a physicist and to being easily confused by the subject, so I'll just use a silly little metaphor to check whether I'm thinking straight, and try to avoid terminology I don't fully understand…
Only locally, though. The next galaxy cluster over has a uniquely privileged inertial reference frame, too, but it's in motion (and accelerating!) relative to ours.
AIT applies to: - Votes with at least three options. A simple majority vote between two options does in fact satisfy all of Arrow's fairness criteria (if only vacuously). - Ranked voting systems, i.e. ones where voters…
Just for the sake of having another solution for the same problem, here's the basically-zero-thought-required brute-force search solution in C: https://gist.github.com/arirahikkala/ef1a07dbd5a8ca3a077c03c... Not as fast…
In particular: The unit of GDP is in fact dollars/year (even if people rarely bother reporting the precise unit), whereas market cap is just dollars. They're as incomparable as velocity and position.
You did misunderstand. Read the next sentence: > The purpose of this frame is to allow an endpoint to gracefully stop accepting new streams (perhaps for a reboot or maintenance), while still finishing processing of…
> Fewer words, create a more powerful message. Fewer commas, create a sentence that makes more sense.