Researchers from Penn State have used an automotive radar sensor with a unique approach to detect the vibrations of a cell phone’s earpiece and decode what the person on the other side of the call
Presently, there are various speech and audio decoding modalities available, but detecting and reconstructing speech from the other side of a smartphone line was not yet explored. Hence, Penn State researchers have integrated an automotive radar sensor with a novel processing approach that could detect the vibrations of a cell phone’s earpiece and interpret what the person on the other side of the call was speaking with up to 83% accuracy.
“We use the radar to sense this vibration and reconstruct what was said by the person on the other side of the line,” Basak said, noting that their approach works even when the audio is completely inaudible to both humans and microphones nearby. “This isn’t the first time similar vulnerabilities or attack modalities have been found, but this particular aspect — detecting and reconstructing speech from the other side of a smartphone line — was not yet explored.”
The radar operates in the millimeter-wave (mmWave) spectrum, specifically in the bands of 60 to 64 gigahertz and 77 to 81 gigahertz, which inspired the researchers to name their approach “mmSpy.” This is a subset of the radio spectrum used for 5G, the fifth-generation standard for communication systems across the globe. The radar sensor data is pre-processed via MATLAB and Python modules, these computing platform-language interfaces also eliminate hardware-related and artifact noise from the data. The team then inputs the above data to machine learning modules that are trained to categorize speech and reconstruct audio. If the distance of radar sensing vibrations is from a foot away, the processed speech is 83% accuracy. The accuracy drops as the radar moves far from the phone, down to 43% accurate at six feet.
After the speech is reconstructed, the researchers can further filter, improve or sort keywords as needed. The team envisions refining their proposal to figure out how to protect against this security vulnerability and how to use it for good purposes and avoid its misuse.
“The methodology that we developed can also be used for sensing vibrations in industrial machinery, smart home systems, and building-monitoring systems,” Basak said. “Vibration tracking over time can help assess wear and tear — using our approach could help identify when machinery needs maintenance before it would traditionally be obvious, for example.”
This research could be further used in tracking a user and calling for help if some health parameter changes dangerously. By implementing the correct set of target actions radars can be used in smart homes and industry to enable faster solutions when problems and issues are detected
Click here for the Published Research Paper
