The research team designed a self-driving bicycle experiment to evaluate the chip’s capability for integrating multimodal information and making prompt decisions. Equipped with the Tianjic chip and IMU sensor and a camera the bicycle was tasked with performing functions such as real-time object detection, tracking, voice-command recognition, riding over a speed bump, obstacle avoidance, balance control and decision making.
The research team developed a variety of neural networks (CNN, CANN, SNN and MLP networks) to enable each task. The models were pretrained and programmed onto the Tianjic chip, which can process the models in parallel and enable seamless on-chip communication across different models.
There is very little information about how the two types of networks, neural and neuromorphic, are trained, which is an important issue for either one separately, and even more important when they're combined. Pei and colleagues write, in the "Methods" section of the paper, that they trained the deep learning part in the normal way, and that for the neuromorphic part, they relied on a method introduced last year by some of the researchers, called "Spatio-temporal backpropagation," a version of the backpropogation approach common in deep learning.
There are also some missing details about the chip's fabrication. For example, the part is said to have "reconfigurable" circuits, but how the circuits are to be reconfigured is never specified. It could be so-called "field-programmable gate array," or FPGA, technology or something else. Code for the project is not provided by the authors as it often is for such research; the authors offer to provide the code "on reasonable request."
More important is the fact the chip may have a hard time stacking up to a lot of competing chips out there, says analyst Gwennap. The specs seem underwhelming, in his view. "Tianjic's reported 1.28 TOPS/watt [trillions of operations per watt, a common measure of performance] is similar to today's GPUs," he notes, referring to graphics processing unit chips made by Nvidia and Advanced Micro Devices. However, the performance is "well behind more advanced architectures" of newer chips, he notes.
Gwennap's colleague, Mike Demler, concurs. He notes some inaccuracies in the paper, such as the contention that spiking neurons require "extra high-precision memory" circuits for some functions. Demler's review of a neuromorphic chip by chip giant Intel, called "Loihi," shows that such is not the case. A chip developed by startup Brainchip, Inc., also proves the claim false, he says. Moreover, since the Loihi chip has already shown that conventional neural networks, such as a convolutional neural network, or CNN, can be implemented as a spiking neuron, there's no need for the kind of "unified" chip that the Tsinghua authors claim.
3 comments
[ 4.6 ms ] story [ 29.9 ms ] threadThe research team developed a variety of neural networks (CNN, CANN, SNN and MLP networks) to enable each task. The models were pretrained and programmed onto the Tianjic chip, which can process the models in parallel and enable seamless on-chip communication across different models.
Video of the unmanned bike: https://youtu.be/VjSs6KCLTC0
There is very little information about how the two types of networks, neural and neuromorphic, are trained, which is an important issue for either one separately, and even more important when they're combined. Pei and colleagues write, in the "Methods" section of the paper, that they trained the deep learning part in the normal way, and that for the neuromorphic part, they relied on a method introduced last year by some of the researchers, called "Spatio-temporal backpropagation," a version of the backpropogation approach common in deep learning.
There are also some missing details about the chip's fabrication. For example, the part is said to have "reconfigurable" circuits, but how the circuits are to be reconfigured is never specified. It could be so-called "field-programmable gate array," or FPGA, technology or something else. Code for the project is not provided by the authors as it often is for such research; the authors offer to provide the code "on reasonable request."
More important is the fact the chip may have a hard time stacking up to a lot of competing chips out there, says analyst Gwennap. The specs seem underwhelming, in his view. "Tianjic's reported 1.28 TOPS/watt [trillions of operations per watt, a common measure of performance] is similar to today's GPUs," he notes, referring to graphics processing unit chips made by Nvidia and Advanced Micro Devices. However, the performance is "well behind more advanced architectures" of newer chips, he notes.
Gwennap's colleague, Mike Demler, concurs. He notes some inaccuracies in the paper, such as the contention that spiking neurons require "extra high-precision memory" circuits for some functions. Demler's review of a neuromorphic chip by chip giant Intel, called "Loihi," shows that such is not the case. A chip developed by startup Brainchip, Inc., also proves the claim false, he says. Moreover, since the Loihi chip has already shown that conventional neural networks, such as a convolutional neural network, or CNN, can be implemented as a spiking neuron, there's no need for the kind of "unified" chip that the Tsinghua authors claim.
I am dying to know where they are harvesting these from?
(yes, there is a pun hiding in the sentence above)