DreamX: Video Diffusion Model as a Universal Latent Action Learner
Automatically prepared brief
DreamX uses a pretrained video diffusion model to learn a universal action representation that can be transferred across different embodiments and environments.
The research aims to develop a universal action representation from video that captures consistent dynamics across action executions while being invariant to changes in appearance and embodiment.
DreamX repurposes a pretrained video diffusion model to extract compact latent actions from video, significantly reducing the dimensionality of video frames and capturing action dynamics independent of visual appearance.
On LARY-Bench, DreamX demonstrates state-of-the-art performance among latent action models, supporting any-to-any video transfer and zero-shot action transfer across various embodiments and environments.
About this brief
This article summarizes a public preprint abstract. Its findings have not been independently reproduced for this publication. Consult the paper for methods, evaluation scope and limitations. Automated briefs may contain errors; the original source takes precedence.