I wonder why not glue it to the top? Somewhere unobtrusive.
Where are you getting 150-400 per million input? https://developers.openai.com/api/docs/pricing https://platform.claude.com/docs/en/about-claude/pricing OpenAI and Mythos are both 10/mil in.…
Well, I'm under the impression that the relations are recursive but you skip computing them recursively by caching B's relations. However if you look at your sibling comments you'll see that I may not have the right…
Oh, of course, but I mean for moving people who want to move around. Someone else observed that ocean liners are less efficient than planes, so if you wanna go overseas there's no alternative. Except, of course, for…
I was under the impression that each new token attends to one previous token per attention head, and that the slowdown observed was more because those attended to tokens are more spread out in memory and get less…
Yes, except no with the KV cache. Because tokens aren't modified by future tokens you can cache the meaning of previous tokens. This makes the total effort linear over the entire context (or constant per forward pass).
Right! When I wrote that I made a mistake and thought that there were filesystems that used multiple drives but still had the meaningful distinction of "what drive is that file on?" Like, yeah, RAID, several drives, one…
Right? If you move between file system or device I would expect "move" to be impossible. At best you could copy a logical block and then delete a logical block. If you share a filesystem then perhaps a logical block is…
Well, it's hard to convey, but he gave off a strong sense of "hitting on her" and she gave off a strong sense of "I'm in customer service and cannot tell him to stop". But, I was in my latest teens, earliest twenties; I…
Another way to think of this is that everybody occasionally needs to do something unusual. But some people, due to their life circumstances, always need to do something unusual. And making things better for the…
This reminded me of something that happened to me maybe 15 years ago. I was helping my Grandpa get his first smart phone. The saleswoman was maybe mid twenties and had a kinda hoarse voice. I was a little bit younger…
There's a tiny bit of play, like sliding window attention. As tokens leave the sliding window you can keep them or discard them. If you keep them, you can freely truncate the context and resume generation from an…
Most pi pico's come with 16mb of flash. I wonder what kind of performance that can eek out.
This author also has other things like: A compiler that emits only `mov` instructions and another compiler that deliberately messes with the control flow so that, if disassembled, common debuggers will draw symbols like…
I've mostly been running LLMs on my own hardware so the phrase "been at it for about 20 minutes" gave me pause. My first instinct was to ask "but on what hardware?" but I suppose one advantage of closed models is that…
I've usually been more annoyed at the surprise dissonance of "was that an anime girl on kernel.org?" than annoyed at the delay.
On one hand, sure, the entire point of a programming language is to make complex ideas able to be expressed in simpler abstractions. On the other hand, we can damn well try.
In the USA you'll get buttloads of mail urging you to do things such as confirm your home warranty at risk of not being covered. With addresses that says "RE: (your mortgage provider)" to make it look like it's from…
You'll see some scripting languages (ab)use this. Where the native "number" type is a 64 bit float and only one NaN bit pattern is a real NaN. The others smuggle a pointer to an object in the lower bits. This way you…
One of my favorite clips to give a sense of scale for rockets is this one. https://www.youtube.com/watch?v=70u748VALt4 I show someone and then I tell them, that's not the rocket exhaust. That's the exhaust for the…
Going by Fabien Sanglard's cheat sheet (who I trust uncritically) https://fabiensanglard.net/usbcheat/index.html it looks like 3.2 actually is a broader term than expected. Maybe there was some awful attempt at…
It reminds me of how vinyl records are fairly lossy, but they provide a superior experience in some cases because those limitations have been accounted for during the mastering process. It's an entire pipeline from…
The term I've seen a lot is https://en.wikipedia.org/wiki/Unit_in_the_last_place So I'd probably rewrite that code to first find the ulp of the larger of the abs of a and b and then assert that their difference is less…
There should be an opposite thruster for each axis. I wonder if the short bursts were due to heating limits.
You can't do software updates securely, but it strikes me that compromising the revocation process is a good thing. Suppose you can use a key to sign a message saying "stop using this". If someone else breaks that key…
I wonder why not glue it to the top? Somewhere unobtrusive.
Where are you getting 150-400 per million input? https://developers.openai.com/api/docs/pricing https://platform.claude.com/docs/en/about-claude/pricing OpenAI and Mythos are both 10/mil in.…
Well, I'm under the impression that the relations are recursive but you skip computing them recursively by caching B's relations. However if you look at your sibling comments you'll see that I may not have the right…
Oh, of course, but I mean for moving people who want to move around. Someone else observed that ocean liners are less efficient than planes, so if you wanna go overseas there's no alternative. Except, of course, for…
I was under the impression that each new token attends to one previous token per attention head, and that the slowdown observed was more because those attended to tokens are more spread out in memory and get less…
Yes, except no with the KV cache. Because tokens aren't modified by future tokens you can cache the meaning of previous tokens. This makes the total effort linear over the entire context (or constant per forward pass).
Right! When I wrote that I made a mistake and thought that there were filesystems that used multiple drives but still had the meaningful distinction of "what drive is that file on?" Like, yeah, RAID, several drives, one…
Right? If you move between file system or device I would expect "move" to be impossible. At best you could copy a logical block and then delete a logical block. If you share a filesystem then perhaps a logical block is…
Well, it's hard to convey, but he gave off a strong sense of "hitting on her" and she gave off a strong sense of "I'm in customer service and cannot tell him to stop". But, I was in my latest teens, earliest twenties; I…
Another way to think of this is that everybody occasionally needs to do something unusual. But some people, due to their life circumstances, always need to do something unusual. And making things better for the…
This reminded me of something that happened to me maybe 15 years ago. I was helping my Grandpa get his first smart phone. The saleswoman was maybe mid twenties and had a kinda hoarse voice. I was a little bit younger…
There's a tiny bit of play, like sliding window attention. As tokens leave the sliding window you can keep them or discard them. If you keep them, you can freely truncate the context and resume generation from an…
Most pi pico's come with 16mb of flash. I wonder what kind of performance that can eek out.
This author also has other things like: A compiler that emits only `mov` instructions and another compiler that deliberately messes with the control flow so that, if disassembled, common debuggers will draw symbols like…
I've mostly been running LLMs on my own hardware so the phrase "been at it for about 20 minutes" gave me pause. My first instinct was to ask "but on what hardware?" but I suppose one advantage of closed models is that…
I've usually been more annoyed at the surprise dissonance of "was that an anime girl on kernel.org?" than annoyed at the delay.
On one hand, sure, the entire point of a programming language is to make complex ideas able to be expressed in simpler abstractions. On the other hand, we can damn well try.
In the USA you'll get buttloads of mail urging you to do things such as confirm your home warranty at risk of not being covered. With addresses that says "RE: (your mortgage provider)" to make it look like it's from…
You'll see some scripting languages (ab)use this. Where the native "number" type is a 64 bit float and only one NaN bit pattern is a real NaN. The others smuggle a pointer to an object in the lower bits. This way you…
One of my favorite clips to give a sense of scale for rockets is this one. https://www.youtube.com/watch?v=70u748VALt4 I show someone and then I tell them, that's not the rocket exhaust. That's the exhaust for the…
Going by Fabien Sanglard's cheat sheet (who I trust uncritically) https://fabiensanglard.net/usbcheat/index.html it looks like 3.2 actually is a broader term than expected. Maybe there was some awful attempt at…
It reminds me of how vinyl records are fairly lossy, but they provide a superior experience in some cases because those limitations have been accounted for during the mastering process. It's an entire pipeline from…
The term I've seen a lot is https://en.wikipedia.org/wiki/Unit_in_the_last_place So I'd probably rewrite that code to first find the ulp of the larger of the abs of a and b and then assert that their difference is less…
There should be an opposite thruster for each axis. I wonder if the short bursts were due to heating limits.
You can't do software updates securely, but it strikes me that compromising the revocation process is a good thing. Suppose you can use a key to sign a message saying "stop using this". If someone else breaks that key…