Researchers at the Massachusetts Institute of Technology have created a robotic system that can find a targeted object in a pile of stuff.Â
The system is known as RFusion, and it consists of a robotic arm with a camera and radio frequency (RF) antenna attached to its gripper. It combines or fuses signals from the antenna with visual input from the camera to locate and retrieve an item, even if the item is buried under a pile and completely out of view.
The researchers, in their prototype, relied on battery-less RFID tags that can be stuck to an item and reflect signals sent by an antenna. RFusion is able to locate a tagged item within a pile as RF signals can travel through most surfaces.
Using machine learning algorithms, the robot narrows down the object’s exact location, moves the items on top of it, grasps the object, and verifies that it picked up the right thing. The camera, antenna, and AI are integrated on the robot.Â
The robot begins by searching for an object using its antenna, which bounces signals off the RFID tag to identify a spherical area in which the tag is located. Then with the help of the camera input, it narrows down the object’s location.Â
“Sometimes, if you only rely on RF measurements, there is going to be an outlier, and if you rely only on vision, there is sometimes going to be a mistake from the camera. But if you combine them, they are going to correct each other. That is what made the system so robust,” Boroushaki says.
The researchers, moreover, used reinforcement learning to train a neural network that can optimize the robot’s trajectory to the object. In reinforcement learning, the algorithm is trained through trial and error with a reward system. In this case, the optimization algorithm was rewarded when it limited the number of moves it had to make to localize the item and the distance it had to travel to pick it up.
The researchers believe that their robot could have many broader applications in the future, like sorting through piles to fulfill orders in a warehouse, identifying and installing components in an auto manufacturing plant, or helping an elderly individual perform daily tasks in the home, though the current prototype isn’t quite fast enough yet for these uses.
“This idea of being able to find items in a chaotic world is an open problem that we’ve been working on for a few years. Having robots that are able to search for things under a pile is a growing need in industry today. Right now, you can think of this as a Roomba on steroids, but in the near term, this could have a lot of applications in manufacturing and warehouse environments,” said senior author Fadel Adib, associate professor in the Department of Electrical Engineering and Computer Science and director of the Signal Kinetics group in the MIT Media Lab.
