New York University researchers have introduced Dobb-E, a framework for training mobile robots in household chores, enabling broader future applications.
For years, roboticists have endeavoured to create robots capable of handling everyday household tasks like dishwashing or cleaning. Despite their efforts, these robots have yet to achieve widespread commercial adoption thus far.
New York University researchers have unveiled Dobb-E, a novel framework tailored for the efficient training of mobile robots in household chores, paving the way for their broader application in the future.
The Dobb-E framework consists of four essential components: a data collection tool, a pre-trained model, a diverse dataset, and a deployment scheme. The “Stick” initial component streamlines the data collection process by utilising a user’s smartphone.
Utilising the Stick data collection tool, the team assembled a novel dataset for training household robots named the “Homes of New York” (HoNY) dataset. This extensive dataset comprises footage gathered from 216 diverse home environments in New York, captured using their smartphone-based setup.
Unlike previous datasets for robot training, the HoNY dataset emphasises a broader array of scenes and robot behaviours. Moreover, the Stick collection tool enabled them to amass an order of magnitude more scenes compared to previous datasets.
The third integral component within the Dobb-E system involves a pre-trained perception model. This model underwent training using the HoNY dataset, adopting a self-supervised learning approach.
The team conducted a series of experiments in actual household settings to evaluate the effectiveness of their data collection tool, HoNY dataset, and pre-trained visual recognition model. They implemented their trained algorithm on the Hello Robot Stretch, a versatile mobile home robot with multiple functions during these experiments.
Remarkably, they trained the robot to perform 109 distinct household tasks. For each task, the researchers fine-tuned their model by incorporating an average of five minutes of new video data.
This recent research holds the potential to shape the advancement of more sophisticated household robot systems. The team has made their data collection tool, dataset, and pre-trained model publicly available, enabling other research teams to employ or modify them for their investigations.
Reference: Nur Muhammad Mahi Shafiullah et al, On Bringing Robots Home, arXiv (2023). DOI: 10.48550/arxiv.2311.16098
