Pixel-UMI is a pipeline for learning environment-aware whole-body policies from sensorized human demonstrations, addressing challenges like embodiment and kinematic differences.
The research aims to develop a method for robots to learn environment-aware whole-body manipulation policies from human demonstrations, despite differences in embodiment and kinematics.
The authors propose Pixel-UMI, which includes Environment-Aware Retargeting (EAR) to determine robot posture and generate collision-aware targets, Delay-calibrated Action Assignment to convert trajectories into actions, and Retargeting-Error-Aware Masking (REAM) to handle viewpoint mismatches. They also use DAgger to collect human corrections for improved performance.
The authors evaluate Pixel-UMI on five challenging tasks, including articulated-object manipulation and contact-rich bimanual coordination, achieving 87.0% average task progress.
About this brief
This article summarizes a public preprint abstract. Its findings have not been independently reproduced for this publication. Consult the paper for methods, evaluation scope and limitations. Automated briefs may contain errors; the original source takes precedence.