RPMArt

Official implementation for the paper RPMArt: Towards Robust Perception and Manipulation for Articulated Objects, accepted by IROS 2024.

For more information, please visit our project website.

---

🛠 Installation

💻 Server-side

1. Clone this repo.

   git clone [email protected]:R-PMArt/rpmart.git
   cd rpmart

2. Create a Conda environment.

   conda create -n rpmart python=3.8
   conda activate rpmart

3. Install PyTorch.

   pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 --extra-index-url https://download.pytorch.org/whl/cu113

4. Install pytorch-gradual-warmup-lr.

   cd deps
   git clone [email protected]:ildoonet/pytorch-gradual-warmup-lr.git
   cd pytorch-gradual-warmup-lr
   pip install .
   cd ../..

5. Install MinkowskiEngine.

   conda install openblas-devel -c anaconda
   export CUDA_HOME=/usr/local/cuda
   pip install ninja
   cd deps
   git clone [email protected]:NVIDIA/MinkowskiEngine.git
   cd MinkowskiEngine
   pip install -U . --no-deps --install-option="--blas_include_dirs=${CONDA_PREFIX}/include" --install-option="--blas=openblas"
   cd ../..

6. Install CuPy.

   pip install cupy-cuda11x

7. Install special SAPIEN.

   pip install http://download.cs.stanford.edu/orion/where2act/where2act_sapien_wheels/sapien-0.8.0.dev0-cp38-cp38-manylinux2014_x86_64.whl

8. Install AnyGrasp.

   pip install cvxopt munch graspnetAPI
   # follow AnyGrasp to use the licenses and weights and binary codes

9. Install other dependencies.

   pip install -r requirements.txt

10. Build shot.

    # you may need first install pybind11 and pcl
    cd src_shot
    mkdir build
    cd build
    cmake ..
    make
    cd ../..

🦾 Robot-side

1. Make sure rt-linux is enabled for Franka Emika Panda.

   uname -a

2. Install frankx for robot and pyrealsense2 for camera.

   pip install frankx pyrealsense2

3. Install paramiko for connecting with server.

   pip install paramiko

🏃‍♂️ Run

1. Train or download RoArtNet.

   bash scripts/train.sh

2. Test RoArtNet.

   bash scripts/test.sh

3. Evaluate RPMArt.

   bash scripts/eval_roartnet.sh

4. Test RoArtNet on RealArt-6.

   bash scripts/test_real.sh

5. Evaluate RPMArt in the real world.

   # server side
   bash scripts/real_service.sh
   
   # robot side
   python real_eval.py

🙏 Acknowledgement

• Our simulation environment is adapted from VAT-Mart.
• Our voting module is adapted from CPPF and CPPF++.

✍ Citation

If you find our work useful, please consider citing:

@article{wang2024rpmart,
  title={RPMArt: Towards Robust Perception and Manipulation for Articulated Objects},
  author={Wang, Junbo and Liu, Wenhai and Yu, Qiaojun and You, Yang and Liu, Liu and Wang, Weiming and Lu, Cewu},
  journal={arXiv preprint arXiv:2403.16023},
  year={2024}
}

📃 License

This repository is released under the MIT license.