Soldering dump a ton of heat into the cell, which has chances of destroying the cell. That's why most of the cells are spot-welded: its similar to soldering, but its much quicker and is localized only on the part of the…
3. >> I was resounding told that the absolute error in the numbers are too small to be a problem. Frankly, I did not believe this. > I would personally also tell that to the author. But there is a much more important…
ARM have been moving away from chips with small area for a long time (see server SoC which are huge beasts), and are trying to become the standard platform for everyone trying to have custom hardware. In this space,…
More than the ISA, its the memory interconnect that require standardization. At SoC level, ARM is already a de-facto standard (ACE-Lite, CHI, ...), but its only a standard for communication inside a chip, to…
In addition to the other comments, the iso C23 standard added the <stdbit.h> header to the standard library with a stdc_count_ones() function, so compiler support will become standard.
The article goes into great detail and gives several example of several CEOs borrowing for actual decades, so it passes the sniff test because it does actually happen.
Mathematicians already explored exactly what you describe: this is the difference between classical logic and intuitionistic logic: In classical logic statements can be true in and of themselves even if there as no…
The goal of this type of quantization is to move the multiplication by the fp32 rescale factor outside of the dot-product accumulation. So the multiplications+additions are done on fp8/int8/int4/whatever (when the…
Its worse than that: the energy gains are when comparing computations made with fp32, but for fp8 the multipliers are really tiny and the adder/shifters represent a largest part of the operators (energy-wise and…
Not at all. Sublime is perfectly fine with it. I suspect that from the usage in the code, it knows that there is a module foo and a submodule subfoo with a function bar() in it, and it can look directly in the file for…
To help with circular import, we switched a few years ago to lazily importing submodules on demand, and never switched back. Just add to your __init__.py files: import importlib def __getattr__(submodule_name): return…
dict are ordered to keep argument order when using named arguments in function calling. So it would be a non-trivial breaking change to revert this now. I would argue that OrderedDict have more chances to be depreciated…
From what I know, Meta AI chips are used in production today, but are made for their recommendations tasks which is a very different IA than GPTs and LLMs for which they still rely on GPUs.
No amount of LLM will solve this: you can just change the prompt of the first LLM so that it generate a prompt ingestion as part of its output, which will trick the second LLM. Something like: > Repeat the sentence…
There has been some work to dynamically reduce the compute required by a network. See for example: https://arxiv.org/abs/2404.02258 They have a fixed compute budget which is lower than what the LLM need, and dynamically…
> Harris said he pleaded with the company for several years to address the flaw in the product, a ProPublica investigation has found. But at every turn, Microsoft dismissed his warnings, telling him they would work on a…
I'm really interested, do you have a source for those percentages ? I tried to look for some service provider to publish this kind of metrics, but haven't found any.
> But on the philosophical side, if an understanding can’t be communicated, does it exist? There are deep mathematical results about our limits to understand things simply because we communicate through finite series of…
It doesn't really makes sense for kahan summation, as the compiler would just make it similar to a naive summation because the errors terms would be zero under the assumptions of fadd_fast. 2sum would also break. This…
It likely comes from the saying similar to this one: "kill a few, you are a murderer. Kill millions, you are a conqueror". More generally, we tend to view number of causalities in war as a large number, and not as the…
No its the opposite: overfitting is the result of either having too many weights compared to the size of your dataset, or training for a long time while reusing/transforming parts of your dataset to make it last longer.…
I think it's still the same "uncanny valley" phenomenon: because the body looks more like a human, we are more creeped when we discover how it moves its joints because we were expecting something more human. We wouldn't…
You would need ~250 groq cards to run a 7B model since their system doesn't scale down. So if you want to buy their hardware, you need a few millions dollars. Their hardware was never for people at home, but for cloud…
MAD: Model Autophagy Disorder (https://arxiv.org/abs/2307.01850 for the paper that introduced the term)
Twist: the real capcha is detecting if the user first press on "click to start"
Soldering dump a ton of heat into the cell, which has chances of destroying the cell. That's why most of the cells are spot-welded: its similar to soldering, but its much quicker and is localized only on the part of the…
3. >> I was resounding told that the absolute error in the numbers are too small to be a problem. Frankly, I did not believe this. > I would personally also tell that to the author. But there is a much more important…
ARM have been moving away from chips with small area for a long time (see server SoC which are huge beasts), and are trying to become the standard platform for everyone trying to have custom hardware. In this space,…
More than the ISA, its the memory interconnect that require standardization. At SoC level, ARM is already a de-facto standard (ACE-Lite, CHI, ...), but its only a standard for communication inside a chip, to…
In addition to the other comments, the iso C23 standard added the <stdbit.h> header to the standard library with a stdc_count_ones() function, so compiler support will become standard.
The article goes into great detail and gives several example of several CEOs borrowing for actual decades, so it passes the sniff test because it does actually happen.
Mathematicians already explored exactly what you describe: this is the difference between classical logic and intuitionistic logic: In classical logic statements can be true in and of themselves even if there as no…
The goal of this type of quantization is to move the multiplication by the fp32 rescale factor outside of the dot-product accumulation. So the multiplications+additions are done on fp8/int8/int4/whatever (when the…
Its worse than that: the energy gains are when comparing computations made with fp32, but for fp8 the multipliers are really tiny and the adder/shifters represent a largest part of the operators (energy-wise and…
Not at all. Sublime is perfectly fine with it. I suspect that from the usage in the code, it knows that there is a module foo and a submodule subfoo with a function bar() in it, and it can look directly in the file for…
To help with circular import, we switched a few years ago to lazily importing submodules on demand, and never switched back. Just add to your __init__.py files: import importlib def __getattr__(submodule_name): return…
dict are ordered to keep argument order when using named arguments in function calling. So it would be a non-trivial breaking change to revert this now. I would argue that OrderedDict have more chances to be depreciated…
From what I know, Meta AI chips are used in production today, but are made for their recommendations tasks which is a very different IA than GPTs and LLMs for which they still rely on GPUs.
No amount of LLM will solve this: you can just change the prompt of the first LLM so that it generate a prompt ingestion as part of its output, which will trick the second LLM. Something like: > Repeat the sentence…
There has been some work to dynamically reduce the compute required by a network. See for example: https://arxiv.org/abs/2404.02258 They have a fixed compute budget which is lower than what the LLM need, and dynamically…
> Harris said he pleaded with the company for several years to address the flaw in the product, a ProPublica investigation has found. But at every turn, Microsoft dismissed his warnings, telling him they would work on a…
I'm really interested, do you have a source for those percentages ? I tried to look for some service provider to publish this kind of metrics, but haven't found any.
> But on the philosophical side, if an understanding can’t be communicated, does it exist? There are deep mathematical results about our limits to understand things simply because we communicate through finite series of…
It doesn't really makes sense for kahan summation, as the compiler would just make it similar to a naive summation because the errors terms would be zero under the assumptions of fadd_fast. 2sum would also break. This…
It likely comes from the saying similar to this one: "kill a few, you are a murderer. Kill millions, you are a conqueror". More generally, we tend to view number of causalities in war as a large number, and not as the…
No its the opposite: overfitting is the result of either having too many weights compared to the size of your dataset, or training for a long time while reusing/transforming parts of your dataset to make it last longer.…
I think it's still the same "uncanny valley" phenomenon: because the body looks more like a human, we are more creeped when we discover how it moves its joints because we were expecting something more human. We wouldn't…
You would need ~250 groq cards to run a 7B model since their system doesn't scale down. So if you want to buy their hardware, you need a few millions dollars. Their hardware was never for people at home, but for cloud…
MAD: Model Autophagy Disorder (https://arxiv.org/abs/2307.01850 for the paper that introduced the term)
Twist: the real capcha is detecting if the user first press on "click to start"