A novel device mimics human vision’s hierarchy and advances in-sensor computing with high efficiency and accuracy.
Researchers from Tsinghua University, China have developed an innovative optoelectronic device that emulates the hierarchical functioning of the human visual system, setting up the standards for advanced in-sensor computing. This device integrates sensing, memory, and processing capabilities using a fully integrated array of optoelectronic memristors (OEMs), offering promising applications in computer vision tasks.
The human visual system processes information at multiple levels, from basic sensory input to complex cognitive functions. To replicate this, the researchers engineered a device with 1kb OEM arrays comprising 128 × 8 cells combined with silicon CMOS circuits. These cells exhibit three distinct configurable modes: electronic memristor, dynamic OEM, and non-volatile OEM (NV-OEM), enabling multi-mode functionality.
“The large-scale integration of in-sensor computing with CMOS circuits has remained a challenge,” noted the team. “We address this with a layered structure using Pd/TiOx/ZnO/TiN materials.” The system concurrently processes and stores data using both electrical and optical signals, significantly enhancing energy efficiency and operational flexibility.
Preliminary experiments demonstrated the device’s potential across diverse tasks. It improved image recognition accuracy from 85.7% to 96.1% using the NV-OEM mode and achieved 96.1% accuracy in object tracking through hybrid dynamic and NV-OEM modes. Additionally, human motion recognition using in-sensor reservoir computing showed 91.2% accuracy. This makes the device particularly appealing to industries like autonomous vehicles, robotics, and healthcare, where real-time, low-power visual processing is crucial for applications such as navigation, diagnostics, and human-computer interaction.
“This platform consumes over 20 times less energy than conventional graphics processing units,” the researchers reported, underscoring its efficiency. By modulating oxygen vacancies with optical and electrical operations, the system mimics the hierarchical structure of human vision effectively.
The research introduces a cost-effective solution for real-time data processing in fields requiring rapid visual interpretation. Future studies aim to further optimise the device by incorporating transparent materials to enhance light absorption, opening new possibilities for its application in energy-efficient computing systems.
