Chinese scientists developed a multimode-fused spiking neuron (MFSN) array, which can sense different shapes, temperatures and weights just like human beings' multi-sensory perception, making a major breakthrough and contributing to future development of intelligent robotics, the Chinese Academy of Sciences (CAS) announced Thursday.

The research, conducted by scientists from the Institute of Microelectronics (IME) of the CAS and Fudan University, was published in the material science academic journal Advanced Materials. 

How will MFSN help improve robotics?

According to the research, multimode-fused sensing in the somatosensory system helps people obtain comprehensive object properties and make accurate judgments, as the MFSN can achieve human-like multi-sensory perception which helps people ascertain the properties of objects and make accurate judgments.

However, building such multi-sensory systems with conventional metal, oxide, semiconductor technology presents serious device integration and circuit complexity challenges. The new MFSN developed by the Chinese scientists, with a compact structure to achieve human-like multi-sensory perception, heterogeneously integrates a pressure sensor to process pressure and a NbOx(Niobium Oxide)-based memristor to sense temperature, said the researchers. 

Memristors can mimic the functions of biological synapse, where it can simultaneously store the weight and modulate the transmitted signal.

With this new MFSN, multi-sensory analog information can be fused into one spike train, showing excellent data compression and conversion capabilities. Both pressure and temperature information are distinguished from fused spikes by decoupling the output frequencies and amplitudes, supporting multimodal tactile perception, according to the research. 

In this case, a 3 × 3 MFSN array is fabricated and the fused frequency patterns are fed into a spiking neural network (SNN) for enhanced pattern recognition. A larger MFSN array is simulated for classifying objects with different shapes, temperatures, and weights, validating the feasibility of the MFSNs for practical applications, said the researchers. 

Therefore, the proof-of-concept MFSNs enable the building of multimodal sensory systems and contribute to the future development of highly intelligent robotics, as the scientific findings can further help to construct multimodal sensory systems and be utilized for practical applications, the researchers added.