Quantization in this context is the precision of each value in the vector or matrix/tensor. If the model in question has a token embedding length of 1024, even if it was a 1 bit quantization, each token has 2^1024…
This is my favorite story of his. It is part of the collection Axiomatic for those new to him.
Anyone who would be on this site is the kind of person who should do themselves a favor and read his short story collection Axiomatic. His other collections are excellent as well, I have read them all. For novels,…
Mike Acton's talk "Data-Oriented Design and C++" from CppCon 2014 is the best programming talk ever given in my opinion. A must watch: https://youtu.be/rX0ItVEVjHc
But maybe you get a few more seconds, one more swing, and take out your opponent. Your tribe kills the lion or wins the war. If the initial mutation wasn't detrimental so wasn't tested right away on an individual basis,…
A theory I heard that I like, is that if you are getting attacked from the right, and suffer damage to the right side of your head, you probably want your right arm to work more than your left for the highest chance of…
You start a bank, I come to take out a loan and I want it in cash since I am buying a used car this afternoon. You have no cash since you said you don't need it, are just going to create it. Where does the cash come…
It is (was) a thing, the idea works exactly the same way. Person A deposits $100, bank lends $90 to B, $10 goes to reserve (if 10% reserve rate). System thinks there is $190 instead of $100, so money is "created". As of…
There are many problems with your understanding, but the simplest total failure of your model is that if the bank did just get the money from somewhere else to lend, it is not creating it. You are not describing the…
The loan guy absolutely does need to make sure the bank has the cash to make the loan. You can't loan more money than you have. What happens when B goes to withdraw it from the bank to buy a car or house if it isn't…
Quantization in this context is the precision of each value in the vector or matrix/tensor. If the model in question has a token embedding length of 1024, even if it was a 1 bit quantization, each token has 2^1024…
This is my favorite story of his. It is part of the collection Axiomatic for those new to him.
Anyone who would be on this site is the kind of person who should do themselves a favor and read his short story collection Axiomatic. His other collections are excellent as well, I have read them all. For novels,…
Mike Acton's talk "Data-Oriented Design and C++" from CppCon 2014 is the best programming talk ever given in my opinion. A must watch: https://youtu.be/rX0ItVEVjHc
But maybe you get a few more seconds, one more swing, and take out your opponent. Your tribe kills the lion or wins the war. If the initial mutation wasn't detrimental so wasn't tested right away on an individual basis,…
A theory I heard that I like, is that if you are getting attacked from the right, and suffer damage to the right side of your head, you probably want your right arm to work more than your left for the highest chance of…
You start a bank, I come to take out a loan and I want it in cash since I am buying a used car this afternoon. You have no cash since you said you don't need it, are just going to create it. Where does the cash come…
It is (was) a thing, the idea works exactly the same way. Person A deposits $100, bank lends $90 to B, $10 goes to reserve (if 10% reserve rate). System thinks there is $190 instead of $100, so money is "created". As of…
There are many problems with your understanding, but the simplest total failure of your model is that if the bank did just get the money from somewhere else to lend, it is not creating it. You are not describing the…
The loan guy absolutely does need to make sure the bank has the cash to make the loan. You can't loan more money than you have. What happens when B goes to withdraw it from the bank to buy a car or house if it isn't…