AutoURDF: Unsupervised Robot Modeling from Point Cloud Frames Using Cluster Registration

Jiong Lin, Lechen Zhang, Kwansoo Lee, Jialong Ning, Judah Goldfeder, Hod Lipson
Columbia University, Creative Machines Lab
CVPR 2025
arXiv Supp Code

Overview

Our method provides a complete pipeline for the automated construction of robot description files. (A). Data Collection: By commanding robots with randomly sampled motor angle sequences, we capture the corresponding time-series point cloud frames. (B). Three Substeps: We tackle the problem in three substeps: 1) part segmentation, 2) body topology inference, and 3) joint parameter estimation. (C). Description File Generation: The final output is a URDF file that defines the robot's links, joints, and collision properties. We successfully build and simulate the description model for the WX200 robot arm in PyBullet with new motor configurations.

Abstract

Robot description models are essential for simulation and control, yet their creation often requires significant manual effort. To streamline this modeling process, we introduce AutoURDF, an unsupervised approach for constructing description files for unseen robots from point cloud frames. Our method leverages a cluster-based point cloud registration model that tracks the 6-DoF transformations of point clusters. Through analyzing cluster movements, we hierarchically address the following challenges: (1) moving part segmentation, (2) body topology inference, and (3) joint parameter estimation. The complete pipeline produces robot description files that are fully compatible with existing simulators. We validate our method across a variety of robots, using both synthetic and real-world scan data. Results indicate that our approach outperforms previous methods in registration and body topology estimation accuracy, offering a scalable solution for automated robot modeling.

Video

Interactive URDF Viewer

Explore and manipulate the generated robot models by our method

Robots
Real-world Scan, WX200
Input
Input
Output
WX200
Input
Input
Output
PhantomX
Input
Input
Output
Pandas
Input
Input
Output
Sapien Dataset
Laptop
Input
Input
Output
Trashcan
Input
Input
Output
Joint Estimation
Joint Estimation

Wx200 Real-World Scan

WX200

PhantomX

Pandas

UR5e

Bolt

OP3

Solo

Allegro

WX200 Real-World WX200 PhantomX Pandas UR5e Bolt OP3 Solo Allegro

BibTeX

@article{lin2024autourdf,
      title={AutoURDF: Unsupervised Robot Modeling from Point Cloud Frames Using Cluster Registration},
      author={Lin, Jiong and Zhang, Lechen and Lee, Kwansoo and Ning, Jialong and Goldfeder, Judah and Lipson, Hod},
      journal={arXiv preprint arXiv:2412.05507},
      year={2024}
    }