This result could lead to task-specific neural networks implemented at the hardware circuit level. This is one step further than quantization and should allow to further reduce energy consumption and accelerate inference time for some specific neural networks.
Would this be even more performant than a dedicated tensor accelerator (such as TPU) ?
Yes from a hardware point of view, custom look-up tables are much cheaper than multipliers. On the other hand, the point seems to be to utilize the LUTs of FPGAs which are not very power-efficient because they also need to be fully programmable. In theory you could hard-code the weights of a network into an ASIC, or even just a new generic building block that is cheaper than multiplication. This would then be near impossible to beat, if anybody can be bothered to optimize for this architecture.
I find it hard to think task specific neural networks will beat a lot of the current task specific hardware. Unlesqs you broaden the definition to a useless degree.
I say this without diminishing what neural networks can and are doing. And I fully ack that I am not an authority in this. It would just surprise me.
Seems like a common theme with the explosion of new programming related projects. Everyone wants to be innovative, why check first when it's more fun to reinvent the wheel (now done in Rust :P)
I think this this pretty unnecessarily (and uncritically) dismissive of this work.
Sure, this work hasn't sprung from the primordial academic goop without influence, but the authors of this paper embraced old ideas to get faster and sparser FPGA implentations of neural nets. We have something now that we didn't before.
The vague ideas were there, but parallel progress in other domains have brought us to the point where we're able to take principles, put them into practice, and learn from them.
10 comments
[ 4.4 ms ] story [ 35.1 ms ] threadThis result could lead to task-specific neural networks implemented at the hardware circuit level. This is one step further than quantization and should allow to further reduce energy consumption and accelerate inference time for some specific neural networks.
Would this be even more performant than a dedicated tensor accelerator (such as TPU) ?
I say this without diminishing what neural networks can and are doing. And I fully ack that I am not an authority in this. It would just surprise me.
It's hilarious; like AI found a whole new crop of people, but ones that aren't serious enough to do the background reading.
Sure, this work hasn't sprung from the primordial academic goop without influence, but the authors of this paper embraced old ideas to get faster and sparser FPGA implentations of neural nets. We have something now that we didn't before.
The vague ideas were there, but parallel progress in other domains have brought us to the point where we're able to take principles, put them into practice, and learn from them.