Well, I'm under the impression that the relations are recursive but you skip computing them recursively by caching B's relations. However if you look at your sibling comments you'll see that I may not have the right…
Oh, of course, but I mean for moving people who want to move around. Someone else observed that ocean liners are less efficient than planes, so if you wanna go overseas there's no alternative. Except, of course, for…
I was under the impression that each new token attends to one previous token per attention head, and that the slowdown observed was more because those attended to tokens are more spread out in memory and get less…
What's the alternative?
Yes, except no with the KV cache. Because tokens aren't modified by future tokens you can cache the meaning of previous tokens. This makes the total effort linear over the entire context (or constant per forward pass).
Right! When I wrote that I made a mistake and thought that there were filesystems that used multiple drives but still had the meaningful distinction of "what drive is that file on?" Like, yeah, RAID, several drives, one…
Right? If you move between file system or device I would expect "move" to be impossible. At best you could copy a logical block and then delete a logical block. If you share a filesystem then perhaps a logical block is…
Well, it's hard to convey, but he gave off a strong sense of "hitting on her" and she gave off a strong sense of "I'm in customer service and cannot tell him to stop". But, I was in my latest teens, earliest twenties; I…
Another way to think of this is that everybody occasionally needs to do something unusual. But some people, due to their life circumstances, always need to do something unusual. And making things better for the…
This reminded me of something that happened to me maybe 15 years ago. I was helping my Grandpa get his first smart phone. The saleswoman was maybe mid twenties and had a kinda hoarse voice. I was a little bit younger…
I hadn't read carefully enough to recognize that! That's a wacky coincidence that I love. Thank you for your writing.
You might appreciate the first author on this paper, https://oeis.org/A180632/a180632.pdf https://en.wikipedia.org/wiki/Superpermutation#Lower_bounds,...
There's a tiny bit of play, like sliding window attention. As tokens leave the sliding window you can keep them or discard them. If you keep them, you can freely truncate the context and resume generation from an…
Most pi pico's come with 16mb of flash. I wonder what kind of performance that can eek out.
This author also has other things like: A compiler that emits only `mov` instructions and another compiler that deliberately messes with the control flow so that, if disassembled, common debuggers will draw symbols like…
I've mostly been running LLMs on my own hardware so the phrase "been at it for about 20 minutes" gave me pause. My first instinct was to ask "but on what hardware?" but I suppose one advantage of closed models is that…
I've usually been more annoyed at the surprise dissonance of "was that an anime girl on kernel.org?" than annoyed at the delay.
I'm picturing something like RUST servers, where the admins get to set their own rules and resource multipliers to facilitate the particular part of gameplay that they want.
On one hand, sure, the entire point of a programming language is to make complex ideas able to be expressed in simpler abstractions. On the other hand, we can damn well try.
In the USA you'll get buttloads of mail urging you to do things such as confirm your home warranty at risk of not being covered. With addresses that says "RE: (your mortgage provider)" to make it look like it's from…
You'll see some scripting languages (ab)use this. Where the native "number" type is a 64 bit float and only one NaN bit pattern is a real NaN. The others smuggle a pointer to an object in the lower bits. This way you…
Vaguely reminds me of that planet in Hitchhiker's Guide that started to lose too much mass to tourists leaving. So, > Thus today the net balance between the amount you eat and the amount you excrete while on the planet…
On a metal as soft as copper I imagine that texture'll last about 30 minutes after it's issued to the soldier.
One of my favorite clips to give a sense of scale for rockets is this one. https://www.youtube.com/watch?v=70u748VALt4 I show someone and then I tell them, that's not the rocket exhaust. That's the exhaust for the…
Going by Fabien Sanglard's cheat sheet (who I trust uncritically) https://fabiensanglard.net/usbcheat/index.html it looks like 3.2 actually is a broader term than expected. Maybe there was some awful attempt at…
Well, I'm under the impression that the relations are recursive but you skip computing them recursively by caching B's relations. However if you look at your sibling comments you'll see that I may not have the right…
Oh, of course, but I mean for moving people who want to move around. Someone else observed that ocean liners are less efficient than planes, so if you wanna go overseas there's no alternative. Except, of course, for…
I was under the impression that each new token attends to one previous token per attention head, and that the slowdown observed was more because those attended to tokens are more spread out in memory and get less…
What's the alternative?
Yes, except no with the KV cache. Because tokens aren't modified by future tokens you can cache the meaning of previous tokens. This makes the total effort linear over the entire context (or constant per forward pass).
Right! When I wrote that I made a mistake and thought that there were filesystems that used multiple drives but still had the meaningful distinction of "what drive is that file on?" Like, yeah, RAID, several drives, one…
Right? If you move between file system or device I would expect "move" to be impossible. At best you could copy a logical block and then delete a logical block. If you share a filesystem then perhaps a logical block is…
Well, it's hard to convey, but he gave off a strong sense of "hitting on her" and she gave off a strong sense of "I'm in customer service and cannot tell him to stop". But, I was in my latest teens, earliest twenties; I…
Another way to think of this is that everybody occasionally needs to do something unusual. But some people, due to their life circumstances, always need to do something unusual. And making things better for the…
This reminded me of something that happened to me maybe 15 years ago. I was helping my Grandpa get his first smart phone. The saleswoman was maybe mid twenties and had a kinda hoarse voice. I was a little bit younger…
I hadn't read carefully enough to recognize that! That's a wacky coincidence that I love. Thank you for your writing.
You might appreciate the first author on this paper, https://oeis.org/A180632/a180632.pdf https://en.wikipedia.org/wiki/Superpermutation#Lower_bounds,...
There's a tiny bit of play, like sliding window attention. As tokens leave the sliding window you can keep them or discard them. If you keep them, you can freely truncate the context and resume generation from an…
Most pi pico's come with 16mb of flash. I wonder what kind of performance that can eek out.
This author also has other things like: A compiler that emits only `mov` instructions and another compiler that deliberately messes with the control flow so that, if disassembled, common debuggers will draw symbols like…
I've mostly been running LLMs on my own hardware so the phrase "been at it for about 20 minutes" gave me pause. My first instinct was to ask "but on what hardware?" but I suppose one advantage of closed models is that…
I've usually been more annoyed at the surprise dissonance of "was that an anime girl on kernel.org?" than annoyed at the delay.
I'm picturing something like RUST servers, where the admins get to set their own rules and resource multipliers to facilitate the particular part of gameplay that they want.
On one hand, sure, the entire point of a programming language is to make complex ideas able to be expressed in simpler abstractions. On the other hand, we can damn well try.
In the USA you'll get buttloads of mail urging you to do things such as confirm your home warranty at risk of not being covered. With addresses that says "RE: (your mortgage provider)" to make it look like it's from…
You'll see some scripting languages (ab)use this. Where the native "number" type is a 64 bit float and only one NaN bit pattern is a real NaN. The others smuggle a pointer to an object in the lower bits. This way you…
Vaguely reminds me of that planet in Hitchhiker's Guide that started to lose too much mass to tourists leaving. So, > Thus today the net balance between the amount you eat and the amount you excrete while on the planet…
On a metal as soft as copper I imagine that texture'll last about 30 minutes after it's issued to the soldier.
One of my favorite clips to give a sense of scale for rockets is this one. https://www.youtube.com/watch?v=70u748VALt4 I show someone and then I tell them, that's not the rocket exhaust. That's the exhaust for the…
Going by Fabien Sanglard's cheat sheet (who I trust uncritically) https://fabiensanglard.net/usbcheat/index.html it looks like 3.2 actually is a broader term than expected. Maybe there was some awful attempt at…