- In a breakthrough research, it has been deemed to possible to equip robots with human-like vision and touch sensing to carry out intricate tasks
- This has been made possible by using the sensing properties of an artificial skin and processing power of Intel’s Loihi neuromorphic chip
Researchers from the National University of Singapore (NUS), who are also members of the Intel Neuromorphic Research Community (INRC), have presented new findings that demonstrate event-based vision and touch sensing in combination with Intel’s neuromorphic processing for robotics. The work highlights robots acquiring a sense of touch to significantly improve capabilities and functionality compared to today’s visual-only systems and how neuromorphic processors can outperform traditional architectures in processing such sensory data.
As most of today’s robots operate solely on visual processing, enabling a human-like sense of touch in robotics could significantly improve current functionality and even lead to new use cases. For example, robotic arms fitted with artificial skin could easily adapt to changes in goods manufactured in a factory, using tactile sensing to identify and grip unfamiliar objects with the right amount of pressure to prevent slipping. The ability to feel and better perceive surroundings could also allow for closer and safer human-robotic interaction or allow ease of performing complex surgical tasks with the help of automation by giving surgical robots a sense of touch.
“Making an ultra-fast artificial skin sensor solves about half the puzzle of making robots smarter,” said assistant professor Benjamin Tee from the NUS Department of Materials Science and Engineering and NUS Institute for Health Innovation & Technology. “They also need an artificial brain that can ultimately achieve perception and learning as another critical piece in the puzzle.”
Creating the system
The researchers at NUS developed an artificial skin, which can detect touch more than 1,000 times faster than the human sensory nervous system and identify the shape, texture and hardness of objects 10 times faster than the blink of an eye. After this, the research team began exploring the potential of neuromorphic technology to process sensory data from the artificial skin using Intel’s Loihi neuromorphic research chip. In their initial experiment, the researchers used a robotic hand that was fitted with the artificial skin to read Braille by transferring the tactile data to Loihi through the cloud for converting the micro-bumps felt by the hand into a semantic meaning. Loihi achieved over 92 per cent accuracy in classifying the Braille letters while using 20 times less power than a standard Von Neumann processor.
Building on this work, the NUS team further improved robotic perception capabilities by combining both vision and touch data in a spiking neural network. To do so, they tasked a robot to classify various opaque containers holding differing amounts of liquid using sensory inputs from the artificial skin and an event-based camera. Researchers used the same tactile and vision sensors to test the ability of the perception system to identify rotational slip, which is important for stable grasping.
Once this sensory data was captured, the team sent it to both a GPU and Intel’s Loihi neuromorphic research chip to compare processing capabilities. The results showed that combining event-based vision and touch using a spiking neural network enabled 10 per cent greater accuracy in object classification compared to a vision-only system. Moreover, they demonstrated the efficacy of a neuromorphic technology to power robotic devices, with Loihi processing the sensory data 21 per cent faster than a top-performing GPU, while using 45 times less power.
“We’re excited by these results. They show that a neuromorphic system is a promising piece of the puzzle for combining multiple sensors to improve robot perception. It’s a step toward building power-efficient and trustworthy robots that can respond quickly and appropriately in unexpected situations,” said assistant professor Harold Soh from the Department of Computer Science at the NUS School of Computing.
Mike Davies, director of Intel’s Neuromorphic Computing Lab, said, “This research from the National University of Singapore provides a compelling glimpse to the future of robotics where information is both sensed and processed in an event-driven manner combining multiple modalities. The work adds to a growing body of results showing that neuromorphic computing can deliver significant gains in latency and power consumption once the entire system is re-engineered in an event-based paradigm spanning sensors, data formats, algorithms, and hardware architecture.”
